{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Numerical Optimization\n",
    "\n",
    "This notebook uses the [Optim](https://github.com/JuliaNLSolvers/Optim.jl) package which has routines for unconstrained optimization and for the case with simple bounds on the solution.\n",
    "\n",
    "As alternatives, consider the [NLopt](https://github.com/JuliaOpt/NLopt.jl) or the [JuMP](https://github.com/JuliaOpt/JuMP.jl) packages. They can easily handle various types of constrained optimization problems."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Packages and Extra Functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "printyellow (generic function with 1 method)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using Optim, Dates\n",
    "\n",
    "include(\"printmat.jl\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Plots\n",
    "\n",
    "#pyplot(size=(600,400))\n",
    "gr(size=(480,320))\n",
    "default(fmt = :svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Optimization with One Choice Variable\n",
    "\n",
    "Running \n",
    "```\n",
    "Sol = optimize(x->fn1(x,0.5),x₀,x₁)\n",
    "```\n",
    "finds the `x` value (in the interval `[x₀,x₁]`) that *minimizes* `fn1(x,0.5)`. The solution for the `fn1` function below should be $x=1.1$. The `x->fn1(x,0.5)` syntax makes this a function of `x` only.\n",
    "\n",
    "The output (`Sol`) contains a lot of information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "fn1 (generic function with 1 method)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function fn1(x,c)                    #notice: the function has two arguments\n",
    "  value = 2*(x - 1.1)^2 - c\n",
    "  return value\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"480\" height=\"320\" viewBox=\"0 0 1920 1280\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip9900\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"1920\" height=\"1280\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<path clip-path=\"url(#clip9900)\" d=\"\n",
       "M0 1280 L1920 1280 L1920 0 L0 0  Z\n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip9901\">\n",
       "    <rect x=\"384\" y=\"0\" width=\"1345\" height=\"1280\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<path clip-path=\"url(#clip9900)\" d=\"\n",
       "M175.611 1105.62 L1872.76 1105.62 L1872.76 121.675 L175.611 121.675  Z\n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip9902\">\n",
       "    <rect x=\"175\" y=\"121\" width=\"1698\" height=\"985\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  223.643,1105.62 223.643,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  623.913,1105.62 623.913,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1024.18,1105.62 1024.18,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1424.45,1105.62 1424.45,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1824.72,1105.62 1824.72,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  175.611,1025.15 1872.76,1025.15 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  175.611,814.662 1872.76,814.662 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  175.611,604.175 1872.76,604.175 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  175.611,393.688 1872.76,393.688 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  175.611,183.201 1872.76,183.201 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  175.611,1105.62 1872.76,1105.62 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  175.611,1105.62 175.611,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  223.643,1105.62 223.643,1093.81 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  623.913,1105.62 623.913,1093.81 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1024.18,1105.62 1024.18,1093.81 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1424.45,1105.62 1424.45,1093.81 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1824.72,1105.62 1824.72,1093.81 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  175.611,1025.15 195.976,1025.15 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  175.611,814.662 195.976,814.662 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  175.611,604.175 195.976,604.175 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  175.611,393.688 195.976,393.688 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9900)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  175.611,183.201 195.976,183.201 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 223.643, 1157.22)\" x=\"223.643\" y=\"1157.22\">-1</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 623.913, 1157.22)\" x=\"623.913\" y=\"1157.22\">0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1024.18, 1157.22)\" x=\"1024.18\" y=\"1157.22\">1</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1424.45, 1157.22)\" x=\"1424.45\" y=\"1157.22\">2</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1824.72, 1157.22)\" x=\"1824.72\" y=\"1157.22\">3</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 156.411, 1042.65)\" x=\"156.411\" y=\"1042.65\">0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 156.411, 832.162)\" x=\"156.411\" y=\"832.162\">2</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 156.411, 621.675)\" x=\"156.411\" y=\"621.675\">4</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 156.411, 411.188)\" x=\"156.411\" y=\"411.188\">6</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 156.411, 200.701)\" x=\"156.411\" y=\"200.701\">8</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:84px; text-anchor:middle;\" transform=\"rotate(0, 1024.18, 73.2)\" x=\"1024.18\" y=\"73.2\">the fn1(x,0.5) function</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:66px; text-anchor:middle;\" transform=\"rotate(0, 1024.18, 1234.68)\" x=\"1024.18\" y=\"1234.68\">x</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9900)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:66px; text-anchor:middle;\" transform=\"rotate(-90, 98.8861, 613.647)\" x=\"98.8861\" y=\"613.647\">y</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip9902)\" style=\"stroke:#ff0000; stroke-width:8; stroke-opacity:1; fill:none\" points=\"\n",
       "  223.643,149.523 263.67,235.822 303.697,317.912 343.724,395.793 383.751,469.463 423.778,538.924 463.805,604.175 503.832,665.216 543.859,722.048 583.886,774.67 \n",
       "  623.913,823.082 663.94,867.284 703.967,907.277 743.994,943.059 784.021,974.632 824.048,1002 864.075,1025.15 904.102,1044.09 944.129,1058.83 984.156,1069.35 \n",
       "  1024.18,1075.67 1064.21,1077.77 1104.24,1075.67 1144.26,1069.35 1184.29,1058.83 1224.32,1044.09 1264.35,1025.15 1304.37,1002 1344.4,974.632 1384.43,943.059 \n",
       "  1424.45,907.277 1464.48,867.284 1504.51,823.082 1544.53,774.67 1584.56,722.048 1624.59,665.216 1664.62,604.175 1704.64,538.924 1744.67,469.463 1784.7,395.793 \n",
       "  1824.72,317.912 \n",
       "  \"/>\n",
       "</svg>\n"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = -1:0.1:3\n",
    "\n",
    "plot( x,fn1.(x,0.5),\n",
    "      linecolor = :red,\n",
    "      linewidth = 2,\n",
    "      legend = nothing,\n",
    "      title = \"the fn1(x,0.5) function\",\n",
    "      xlabel = \"x\",\n",
    "      ylabel = \"y\" )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Results of Optimization Algorithm\n",
      " * Algorithm: Brent's Method\n",
      " * Search Interval: [-2.000000, 3.000000]\n",
      " * Minimizer: 1.100000e+00\n",
      " * Minimum: -5.000000e-01\n",
      " * Iterations: 5\n",
      " * Convergence: max(|x - x_upper|, |x - x_lower|) <= 2*(1.5e-08*|x|+2.2e-16): true\n",
      " * Objective Function Calls: 6\n",
      "\n",
      "The minimum is at:      1.100\n",
      "Compare with the plot above\n",
      "\n"
     ]
    }
   ],
   "source": [
    "Sol = optimize(x->fn1(x,0.5),-2.0,3.0)\n",
    "println(Sol)\n",
    "\n",
    "printlnPs(\"\\nThe minimum is at: \", Optim.minimizer(Sol))     #the optimal x value\n",
    "println(\"Compare with the plot above\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## One Choice Variable: Supplying a Starting Guess Instead (extra)\n",
    "\n",
    "If you prefer to give a starting guess `x₀` instead of an interval, then supply it as as a vector `[x₀]`:\n",
    "```\n",
    "Sol = optimize(x->fn1(x[],0.5),[x₀],LBFGS())  \n",
    "```\n",
    "Notice: *(a)* `x[]` to make it a function of the first (and only) element in the vector `x`; *(b)* choose the `LBFGS()` method since the default method does not work in the case of only one choice variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The minimum is at:      1.100\n"
     ]
    }
   ],
   "source": [
    "Solb = optimize(x->fn1(x[],0.5),[0.1],LBFGS())\n",
    "\n",
    "printlnPs(\"The minimum is at: \", Optim.minimizer(Solb))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Several Choice Variables: Unconstrained Optimization\n",
    "\n",
    "In the example below, we choose $(x,y)$ so as to minimize the fairly simple objective function\n",
    "\n",
    "$\n",
    "(x-2)^2 + (4y+3)^2,  \n",
    "$\n",
    "\n",
    "without any constraints. The solution should be $(x,y)=(2,-3/4)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "fn2 (generic function with 1 method)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function fn2(p)\n",
    "    (x,y) = (p[1],p[2])          #unpack the choice variables and get nicer names\n",
    "     L    = (x-2)^2 + (4*y+3)^2\n",
    "    return L\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "nx = 2*41\n",
    "ny = 2*61\n",
    "x = range(1,stop=5,length=nx)\n",
    "y = range(-1,stop=0,length=ny)\n",
    "\n",
    "loss2d = fill(NaN,(nx,ny))      #matrix with loss fn values\n",
    "for i = 1:nx, j = 1:ny\n",
    "    loss2d[i,j] = fn2([x[i];y[j]])\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"480\" height=\"320\" viewBox=\"0 0 1920 1280\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip0300\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"1920\" height=\"1280\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<path clip-path=\"url(#clip0300)\" d=\"\n",
       "M0 1280 L1920 1280 L1920 0 L0 0  Z\n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip0301\">\n",
       "    <rect x=\"384\" y=\"0\" width=\"1345\" height=\"1280\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<path clip-path=\"url(#clip0300)\" d=\"\n",
       "M239.064 1108.82 L1872.76 1108.82 L1872.76 121.675 L239.064 121.675  Z\n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip0302\">\n",
       "    <rect x=\"239\" y=\"121\" width=\"1635\" height=\"988\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,1108.82 239.064,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  647.487,1108.82 647.487,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1055.91,1108.82 1055.91,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1464.33,1108.82 1464.33,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1872.76,1108.82 1872.76,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,1108.82 1872.76,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,911.39 1872.76,911.39 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,713.961 1872.76,713.961 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,516.533 1872.76,516.533 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,319.104 1872.76,319.104 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,121.675 1872.76,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,1108.82 1872.76,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,1108.82 239.064,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,1108.82 239.064,1096.97 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  647.487,1108.82 647.487,1096.97 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1055.91,1108.82 1055.91,1096.97 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1464.33,1108.82 1464.33,1096.97 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,1108.82 1872.76,1096.97 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,1108.82 258.668,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,911.39 258.668,911.39 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,713.961 258.668,713.961 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,516.533 258.668,516.533 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,319.104 258.668,319.104 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,121.675 258.668,121.675 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 239.064, 1160.42)\" x=\"239.064\" y=\"1160.42\">1</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 647.487, 1160.42)\" x=\"647.487\" y=\"1160.42\">2</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1055.91, 1160.42)\" x=\"1055.91\" y=\"1160.42\">3</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1464.33, 1160.42)\" x=\"1464.33\" y=\"1160.42\">4</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1872.76, 1160.42)\" x=\"1872.76\" y=\"1160.42\">5</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 1126.32)\" x=\"219.864\" y=\"1126.32\">-1.0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 928.89)\" x=\"219.864\" y=\"928.89\">-0.8</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 731.461)\" x=\"219.864\" y=\"731.461\">-0.6</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 534.033)\" x=\"219.864\" y=\"534.033\">-0.4</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 336.604)\" x=\"219.864\" y=\"336.604\">-0.2</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 139.175)\" x=\"219.864\" y=\"139.175\">0.0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:84px; text-anchor:middle;\" transform=\"rotate(0, 1055.91, 73.2)\" x=\"1055.91\" y=\"73.2\">Contour plot of loss function</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:66px; text-anchor:middle;\" transform=\"rotate(0, 1055.91, 1237.88)\" x=\"1055.91\" y=\"1237.88\">x</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:66px; text-anchor:middle;\" transform=\"rotate(-90, 94.0861, 615.247)\" x=\"94.0861\" y=\"615.247\">y</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#130a34; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  321.681,1108.82 319.74,1107.89 305.512,1100.66 299.571,1097.54 290.538,1092.5 279.402,1086.07 276.568,1084.34 263.673,1076.19 259.233,1073.27 251.664,1068.03 \n",
       "  240.338,1059.87 239.064,1058.91 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#130a34; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  973.198,1108.82 985.318,1102.84 989.488,1100.66 1004.55,1092.5 1005.49,1091.98 1018.35,1084.34 1025.66,1079.84 1031.29,1076.19 1043.38,1068.03 1045.83,1066.31 \n",
       "  1054.54,1059.87 1065.14,1051.71 1065.99,1051.03 1074.86,1043.55 1084.11,1035.39 1086.16,1033.5 1092.63,1027.24 1100.65,1019.08 1106.33,1013 1108.19,1010.92 \n",
       "  1115.09,1002.76 1121.6,994.604 1126.5,988.072 1127.67,986.445 1133.17,978.287 1138.29,970.129 1143.05,961.971 1146.67,955.219 1147.4,953.813 1151.24,945.654 \n",
       "  1154.73,937.496 1157.87,929.338 1160.64,921.18 1163.06,913.022 1165.11,904.863 1166.81,896.705 1166.84,896.552 1168.11,888.547 1169.05,880.389 1169.66,872.231 \n",
       "  1169.91,864.072 1169.83,855.914 1169.4,847.756 1168.62,839.598 1167.5,831.439 1166.84,827.735 1166.01,823.281 1164.13,815.123 1161.89,806.965 1159.3,798.807 \n",
       "  1156.34,790.648 1153.03,782.49 1149.36,774.332 1146.67,768.875 1145.28,766.174 1140.72,758.016 1135.78,749.857 1130.47,741.699 1126.5,736.007 1124.71,733.541 \n",
       "  1118.39,725.383 1111.69,717.225 1106.33,711.068 1104.51,709.066 1096.69,700.908 1088.47,692.75 1086.16,690.572 1079.54,684.592 1070.08,676.433 1065.99,673.067 \n",
       "  1059.9,668.275 1049.08,660.117 1045.83,657.764 1037.39,651.959 1025.66,644.193 1025.03,643.801 1011.54,635.642 1005.49,632.115 997.085,627.484 985.318,621.22 \n",
       "  981.541,619.326 965.149,611.378 964.687,611.168 946.166,603.01 944.98,602.503 925.794,594.851 924.811,594.471 904.642,587.216 903.069,586.693 884.473,580.69 \n",
       "  877.205,578.535 864.304,574.818 847.381,570.377 844.135,569.548 823.966,564.903 810.899,562.219 803.797,560.799 783.628,557.255 763.459,554.201 762.354,554.06 \n",
       "  743.29,551.698 723.121,549.676 702.952,548.129 682.783,547.058 662.614,546.463 642.445,546.344 622.276,546.701 602.107,547.534 581.938,548.843 561.769,550.628 \n",
       "  541.6,552.888 532.962,554.06 521.431,555.667 501.262,558.966 483.942,562.219 481.093,562.768 460.924,567.163 447.685,570.377 440.755,572.107 420.585,577.659 \n",
       "  417.674,578.535 400.416,583.879 392.041,586.693 380.247,590.775 369.328,594.851 360.078,598.411 348.939,603.01 339.909,606.857 330.433,611.168 319.74,616.193 \n",
       "  313.471,619.326 299.571,626.511 297.794,627.484 283.409,635.642 279.402,637.998 270.06,643.801 259.233,650.781 257.499,651.959 245.944,660.117 239.064,665.175 \n",
       "  \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#240b4e; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,564.4 243.5,562.219 259.233,554.689 260.617,554.06 279.055,545.902 279.402,545.753 299.022,537.744 299.571,537.525 319.74,529.949 320.769,529.586 \n",
       "  339.909,522.989 344.748,521.427 360.078,516.597 371.405,513.269 380.247,510.732 400.416,505.365 401.452,505.111 420.585,500.525 436.876,496.953 440.755,496.121 \n",
       "  460.924,492.199 480.425,488.795 481.093,488.681 501.262,485.63 521.431,482.973 541.6,480.71 542.391,480.636 561.769,478.878 581.938,477.433 602.107,476.374 \n",
       "  622.276,475.699 642.445,475.41 662.614,475.507 682.783,475.988 702.952,476.855 723.121,478.107 743.29,479.745 752.178,480.636 763.459,481.792 783.628,484.252 \n",
       "  803.797,487.106 814.284,488.795 823.966,490.388 844.135,494.11 858.041,496.953 864.304,498.262 884.473,502.891 893.358,505.111 904.642,507.996 923.712,513.269 \n",
       "  924.811,513.58 944.98,519.721 950.219,521.427 965.149,526.41 974.081,529.586 985.318,533.681 995.822,537.744 1005.49,541.578 1015.79,545.902 1025.66,550.151 \n",
       "  1034.26,554.06 1045.83,559.454 1051.46,562.219 1065.99,569.548 1067.56,570.377 1082.55,578.535 1086.16,580.556 1096.63,586.693 1106.33,592.554 1109.97,594.851 \n",
       "  1122.51,603.01 1126.5,605.691 1134.32,611.168 1145.6,619.326 1146.67,620.131 1156.11,627.484 1166.22,635.642 1166.84,636.161 1175.63,643.801 1184.66,651.959 \n",
       "  1187.01,654.159 1193.13,660.117 1201.17,668.275 1207.18,674.633 1208.82,676.433 1215.94,684.592 1222.74,692.75 1227.35,698.554 1229.15,700.908 1235.1,709.066 \n",
       "  1240.74,717.225 1246.07,725.383 1247.52,727.732 1250.97,733.541 1255.53,741.699 1259.79,749.857 1263.75,758.016 1267.41,766.174 1267.68,766.851 1270.67,774.332 \n",
       "  1273.63,782.49 1276.31,790.648 1278.69,798.807 1280.79,806.965 1282.6,815.123 1284.12,823.281 1285.35,831.439 1286.29,839.598 1286.94,847.756 1287.3,855.914 \n",
       "  1287.37,864.072 1287.15,872.231 1286.65,880.389 1285.85,888.547 1284.77,896.705 1283.39,904.863 1281.73,913.022 1279.78,921.18 1277.54,929.338 1275.01,937.496 \n",
       "  1272.19,945.654 1269.08,953.813 1267.68,957.157 1265.61,961.971 1261.8,970.129 1257.7,978.287 1253.29,986.445 1248.58,994.604 1247.52,996.342 1243.44,1002.76 \n",
       "  1237.96,1010.92 1232.16,1019.08 1227.35,1025.52 1226.02,1027.24 1219.38,1035.39 1212.42,1043.55 1207.18,1049.43 1205.07,1051.71 1197.19,1059.87 1188.98,1068.03 \n",
       "  1187.01,1069.91 1180.19,1076.19 1170.98,1084.34 1166.84,1087.88 1161.21,1092.5 1150.92,1100.66 1146.67,1103.91 1140,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#390962; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1263.17,1108.82 1267.68,1104.68 1271.92,1100.66 1280.24,1092.5 1287.85,1084.76 1288.25,1084.34 1295.75,1076.19 1302.97,1068.03 1308.02,1062.08 1309.85,1059.87 \n",
       "  1316.3,1051.71 1322.49,1043.55 1328.19,1035.68 1328.39,1035.39 1333.87,1027.24 1339.08,1019.08 1344.03,1010.92 1348.36,1003.37 1348.7,1002.76 1353,994.604 \n",
       "  1357.03,986.445 1360.82,978.287 1364.34,970.129 1367.61,961.971 1368.53,959.482 1370.57,953.813 1373.25,945.654 1375.68,937.496 1377.86,929.338 1379.79,921.18 \n",
       "  1381.47,913.022 1382.91,904.863 1384.09,896.705 1385.03,888.547 1385.71,880.389 1386.15,872.231 1386.34,864.072 1386.27,855.914 1385.96,847.756 1385.4,839.598 \n",
       "  1384.59,831.439 1383.53,823.281 1382.22,815.123 1380.66,806.965 1378.86,798.807 1376.8,790.648 1374.49,782.49 1371.94,774.332 1369.13,766.174 1368.53,764.565 \n",
       "  1366.01,758.016 1362.61,749.857 1358.96,741.699 1355.05,733.541 1350.88,725.383 1348.36,720.735 1346.4,717.225 1341.59,709.066 1336.51,700.908 1331.16,692.75 \n",
       "  1328.19,688.428 1325.48,684.592 1319.43,676.433 1313.11,668.275 1308.02,661.983 1306.47,660.117 1299.39,651.959 1292.04,643.801 1287.85,639.334 1284.29,635.642 \n",
       "  1276.12,627.484 1267.68,619.352 1267.66,619.326 1258.62,611.168 1249.28,603.01 1247.52,601.515 1239.38,594.851 1229.1,586.693 1227.35,585.34 1218.21,578.535 \n",
       "  1207.18,570.557 1206.92,570.377 1194.9,562.219 1187.01,557.01 1182.37,554.06 1169.2,545.902 1166.84,544.477 1155.24,537.744 1146.67,532.889 1140.6,529.586 \n",
       "  1126.5,522.106 1125.17,521.427 1108.74,513.269 1106.33,512.1 1091.28,505.111 1086.16,502.788 1072.7,496.953 1065.99,494.109 1052.84,488.795 1045.83,486.023 \n",
       "  1031.49,480.636 1025.66,478.492 1008.41,472.478 1005.49,471.482 985.318,464.973 983.169,464.32 965.149,458.953 955.162,456.162 944.98,453.373 924.811,448.21 \n",
       "  923.944,448.004 904.642,443.499 887.706,439.845 884.473,439.161 864.304,435.241 844.232,431.687 844.135,431.67 823.966,428.508 803.797,425.687 786.233,423.529 \n",
       "  783.628,423.215 763.459,421.117 743.29,419.356 723.121,417.929 702.952,416.839 682.783,416.084 662.614,415.664 642.445,415.58 622.276,415.832 602.107,416.419 \n",
       "  581.938,417.342 561.769,418.6 541.6,420.194 521.431,422.124 508.922,423.529 501.262,424.405 481.093,427.055 460.924,430.046 450.996,431.687 440.755,433.412 \n",
       "  420.585,437.158 407.345,439.845 400.416,441.279 380.247,445.808 371.182,448.004 360.078,450.746 339.909,456.09 339.655,456.162 319.74,461.91 311.891,464.32 \n",
       "  299.571,468.18 286.628,472.478 279.402,474.928 263.467,480.636 259.233,482.186 242.087,488.795 239.064,489.985 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#4c0c6a; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,428.166 252.915,423.529 259.233,421.453 278.728,415.371 279.402,415.164 299.571,409.316 307.257,407.213 319.74,403.857 338.743,399.054 339.909,398.765 \n",
       "  360.078,394.074 374.739,390.896 380.247,389.722 400.416,385.738 416.892,382.738 420.585,382.077 440.755,378.774 460.924,375.778 469.912,374.58 481.093,373.114 \n",
       "  501.262,370.772 521.431,368.733 541.6,366.995 549.663,366.421 561.769,365.574 581.938,364.459 602.107,363.642 622.276,363.122 642.445,362.899 662.614,362.973 \n",
       "  682.783,363.345 702.952,364.014 723.121,364.98 743.29,366.243 745.599,366.421 763.459,367.826 783.628,369.715 803.797,371.905 823.966,374.398 825.276,374.58 \n",
       "  844.135,377.237 864.304,380.387 878.023,382.738 884.473,383.862 904.642,387.691 920.253,390.896 924.811,391.848 944.98,396.38 956.104,399.054 965.149,401.268 \n",
       "  985.318,406.527 987.794,407.213 1005.49,412.199 1016.13,415.371 1025.66,418.265 1042.09,423.529 1045.83,424.747 1065.99,431.67 1066.04,431.687 1086.16,439.074 \n",
       "  1088.17,439.845 1106.33,446.962 1108.88,448.004 1126.5,455.365 1128.33,456.162 1146.67,464.313 1146.69,464.32 1163.96,472.478 1166.84,473.869 1180.34,480.636 \n",
       "  1187.01,484.055 1195.92,488.795 1207.18,494.914 1210.8,496.953 1224.95,505.111 1227.35,506.522 1238.4,513.269 1247.52,518.966 1251.32,521.427 1263.65,529.586 \n",
       "  1267.68,532.323 1275.42,537.744 1286.77,545.902 1287.85,546.7 1297.53,554.06 1307.97,562.219 1308.02,562.265 1317.82,570.377 1327.4,578.535 1328.19,579.227 \n",
       "  1336.46,586.693 1345.24,594.851 1348.36,597.845 1353.6,603.01 1361.61,611.168 1368.53,618.448 1369.34,619.326 1376.63,627.484 1383.68,635.642 1388.7,641.671 \n",
       "  1390.42,643.801 1396.79,651.959 1402.92,660.117 1408.8,668.275 1408.87,668.367 1414.3,676.433 1419.56,684.592 1424.58,692.75 1429.04,700.351 1429.36,700.908 \n",
       "  1433.79,709.066 1437.99,717.225 1441.96,725.383 1445.7,733.541 1449.21,741.675 1449.22,741.699 1452.42,749.857 1455.39,758.016 1458.14,766.174 1460.67,774.332 \n",
       "  1462.97,782.49 1465.05,790.648 1466.9,798.807 1468.53,806.965 1469.38,811.89 1469.92,815.123 1471.07,823.281 1472,831.439 1472.71,839.598 1473.2,847.756 \n",
       "  1473.48,855.914 1473.53,864.072 1473.37,872.231 1472.99,880.389 1472.38,888.547 1471.56,896.705 1470.52,904.863 1469.38,912.279 1469.26,913.022 1467.74,921.18 \n",
       "  1466,929.338 1464.04,937.496 1461.85,945.654 1459.43,953.813 1456.8,961.971 1453.93,970.129 1450.84,978.287 1449.21,982.322 1447.49,986.445 1443.86,994.604 \n",
       "  1440.01,1002.76 1435.92,1010.92 1431.6,1019.08 1429.04,1023.68 1427,1027.24 1422.1,1035.39 1416.96,1043.55 1411.58,1051.71 1408.87,1055.66 1405.89,1059.87 \n",
       "  1399.89,1068.03 1393.64,1076.19 1388.7,1082.4 1387.11,1084.34 1380.19,1092.5 1373.02,1100.66 1368.53,1105.6 1365.52,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#5e126e; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1454.96,1108.82 1461.64,1100.66 1468.1,1092.5 1469.38,1090.83 1474.21,1084.34 1480.07,1076.19 1485.71,1068.03 1489.54,1062.26 1491.1,1059.87 1496.18,1051.71 \n",
       "  1501.05,1043.55 1505.7,1035.39 1509.71,1028.02 1510.13,1027.24 1514.26,1019.08 1518.17,1010.92 1521.88,1002.76 1525.38,994.604 1528.67,986.445 1529.88,983.249 \n",
       "  1531.71,978.287 1534.52,970.129 1537.13,961.971 1539.53,953.813 1541.72,945.654 1543.71,937.496 1545.5,929.338 1547.08,921.18 1548.46,913.022 1549.64,904.863 \n",
       "  1550.05,901.387 1550.6,896.705 1551.34,888.547 1551.89,880.389 1552.24,872.231 1552.39,864.072 1552.34,855.914 1552.09,847.756 1551.64,839.598 1551,831.439 \n",
       "  1550.15,823.281 1550.05,822.542 1549.08,815.123 1547.8,806.965 1546.32,798.807 1544.63,790.648 1542.74,782.49 1540.65,774.332 1538.35,766.174 1535.85,758.016 \n",
       "  1533.14,749.857 1530.23,741.699 1529.88,740.779 1527.06,733.541 1523.66,725.383 1520.05,717.225 1516.24,709.066 1512.22,700.908 1509.71,696.08 1507.94,692.75 \n",
       "  1503.4,684.592 1498.64,676.433 1493.67,668.275 1489.54,661.794 1488.45,660.117 1482.92,651.959 1477.17,643.801 1471.2,635.642 1469.38,633.241 1464.9,627.484 \n",
       "  1458.33,619.326 1451.54,611.168 1449.21,608.457 1444.4,603.01 1436.98,594.851 1429.32,586.693 1429.04,586.402 1421.23,578.535 1412.9,570.377 1408.87,566.533 \n",
       "  1404.22,562.219 1395.18,554.06 1388.7,548.365 1385.82,545.902 1376.03,537.744 1368.53,531.644 1365.93,529.586 1355.35,521.427 1348.36,516.165 1344.4,513.269 \n",
       "  1332.99,505.111 1328.19,501.758 1321.11,496.953 1308.81,488.795 1308.02,488.286 1295.87,480.636 1287.85,475.698 1282.46,472.478 1268.5,464.32 1267.68,463.851 \n",
       "  1253.82,456.162 1247.52,452.738 1238.5,448.004 1227.35,442.255 1222.5,439.845 1207.18,432.366 1205.73,431.687 1188.07,423.529 1187.01,423.046 1169.43,415.371 \n",
       "  1166.84,414.258 1149.76,407.213 1146.67,405.961 1128.92,399.054 1126.5,398.129 1106.77,390.896 1106.33,390.738 1086.16,383.784 1082.99,382.738 1065.99,377.237 \n",
       "  1057.38,374.58 1045.83,371.074 1029.69,366.421 1025.66,365.277 1005.49,359.852 999.237,358.263 985.318,354.782 965.47,350.105 965.149,350.031 944.98,345.641 \n",
       "  926.813,341.947 924.811,341.546 904.642,337.792 884.473,334.321 881.107,333.789 864.304,331.174 844.135,328.314 823.966,325.733 823.067,325.63 803.797,323.464 \n",
       "  783.628,321.472 763.459,319.755 743.29,318.312 728.787,317.472 723.121,317.149 702.952,316.269 682.783,315.66 662.614,315.322 642.445,315.254 622.276,315.457 \n",
       "  602.107,315.931 581.938,316.675 566.104,317.472 561.769,317.694 541.6,318.999 521.431,320.579 501.262,322.434 481.093,324.564 472.144,325.63 460.924,326.989 \n",
       "  440.755,329.709 420.585,332.708 413.938,333.789 400.416,336.021 380.247,339.633 368.271,341.947 360.078,343.555 339.909,347.8 329.654,350.105 319.74,352.369 \n",
       "  299.571,357.268 295.703,358.263 279.402,362.527 265.309,366.421 259.233,368.128 239.064,374.096 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#71196d; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,325.456 259.233,320.029 269.243,317.472 279.402,314.915 299.571,310.11 303.113,309.314 319.74,305.631 339.909,301.43 341.313,301.156 360.078,297.548 \n",
       "  380.247,293.933 385.877,292.998 400.416,290.616 420.585,287.571 440.36,284.839 440.755,284.786 460.924,282.294 481.093,280.059 501.262,278.078 517.602,276.681 \n",
       "  521.431,276.358 541.6,274.909 561.769,273.713 581.938,272.768 602.107,272.075 622.276,271.634 642.445,271.445 662.614,271.508 682.783,271.823 702.952,272.39 \n",
       "  723.121,273.209 743.29,274.279 763.459,275.602 777.275,276.681 783.628,277.184 803.797,279.037 823.966,281.145 844.135,283.508 854.388,284.839 864.304,286.145 \n",
       "  884.473,289.061 904.642,292.235 909.119,292.998 924.811,295.708 944.98,299.454 953.542,301.156 965.149,303.497 985.318,307.831 991.817,309.314 1005.49,312.479 \n",
       "  1025.66,317.419 1025.86,317.472 1045.83,322.708 1056.42,325.63 1065.99,328.314 1084.62,333.789 1086.16,334.25 1106.33,340.554 1110.6,341.947 1126.5,347.224 \n",
       "  1134.83,350.105 1146.67,354.27 1157.58,358.263 1166.84,361.71 1179.02,366.421 1187.01,369.563 1199.3,374.58 1207.18,377.851 1218.53,382.738 1227.35,386.597 \n",
       "  1236.83,390.896 1247.52,395.823 1254.29,399.054 1267.68,405.556 1270.99,407.213 1286.97,415.371 1287.85,415.831 1302.2,423.529 1308.02,426.712 1316.85,431.687 \n",
       "  1328.19,438.202 1330.97,439.845 1344.49,448.004 1348.36,450.383 1357.49,456.162 1368.53,463.296 1370.07,464.32 1382.1,472.478 1388.7,477.047 1393.74,480.636 \n",
       "  1404.97,488.795 1408.87,491.694 1415.76,496.953 1426.22,505.111 1429.04,507.363 1436.25,513.269 1445.98,521.427 1449.21,524.202 1455.32,529.586 1464.36,537.744 \n",
       "  1469.38,542.388 1473.08,545.902 1481.46,554.06 1489.54,562.141 1489.62,562.219 1497.37,570.377 1504.92,578.535 1509.71,583.877 1512.18,586.693 1519.13,594.851 \n",
       "  1525.87,603.01 1529.88,608.019 1532.35,611.168 1538.52,619.326 1544.5,627.484 1550.05,635.335 1550.26,635.642 1555.71,643.801 1560.95,651.959 1565.99,660.117 \n",
       "  1570.22,667.236 1570.82,668.275 1575.37,676.433 1579.72,684.592 1583.87,692.75 1587.83,700.908 1590.39,706.468 1591.56,709.066 1595.05,717.225 1598.35,725.383 \n",
       "  1601.46,733.541 1604.38,741.699 1607.1,749.857 1609.64,758.016 1610.56,761.218 1611.95,766.174 1614.06,774.332 1615.98,782.49 1617.71,790.648 1619.26,798.807 \n",
       "  1620.62,806.965 1621.79,815.123 1622.77,823.281 1623.57,831.439 1624.18,839.598 1624.6,847.756 1624.83,855.914 1624.88,864.072 1624.74,872.231 1624.41,880.389 \n",
       "  1623.89,888.547 1623.19,896.705 1622.3,904.863 1621.22,913.022 1619.96,921.18 1618.51,929.338 1616.87,937.496 1615.04,945.654 1613.03,953.813 1610.83,961.971 \n",
       "  1610.56,962.893 1608.4,970.129 1605.77,978.287 1602.94,986.445 1599.93,994.604 1596.73,1002.76 1593.33,1010.92 1590.39,1017.61 1589.73,1019.08 1585.87,1027.24 \n",
       "  1581.82,1035.39 1577.57,1043.55 1573.12,1051.71 1570.22,1056.81 1568.44,1059.87 1563.5,1068.03 1558.35,1076.19 1553.01,1084.34 1550.05,1088.7 1547.41,1092.5 \n",
       "  1541.54,1100.66 1535.46,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#83206b; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1609.27,1108.82 1610.56,1106.98 1614.87,1100.66 1620.26,1092.5 1625.46,1084.34 1630.47,1076.19 1630.73,1075.75 1635.2,1068.03 1639.74,1059.87 1644.1,1051.71 \n",
       "  1648.27,1043.55 1650.9,1038.19 1652.24,1035.39 1655.97,1027.24 1659.52,1019.08 1662.89,1010.92 1666.09,1002.76 1669.1,994.604 1671.07,988.944 1671.92,986.445 \n",
       "  1674.52,978.287 1676.94,970.129 1679.19,961.971 1681.26,953.813 1683.16,945.654 1684.88,937.496 1686.42,929.338 1687.79,921.18 1688.98,913.022 1689.99,904.863 \n",
       "  1690.83,896.705 1691.23,891.72 1691.49,888.547 1691.96,880.389 1692.27,872.231 1692.4,864.072 1692.35,855.914 1692.14,847.756 1691.75,839.598 1691.23,832.171 \n",
       "  1691.18,831.439 1690.43,823.281 1689.51,815.123 1688.41,806.965 1687.13,798.807 1685.67,790.648 1684.04,782.49 1682.23,774.332 1680.25,766.174 1678.09,758.016 \n",
       "  1675.75,749.857 1673.24,741.699 1671.07,735.106 1670.54,733.541 1667.62,725.383 1664.51,717.225 1661.23,709.066 1657.77,700.908 1654.13,692.75 1650.9,685.856 \n",
       "  1650.29,684.592 1646.21,676.433 1641.94,668.275 1637.49,660.117 1632.86,651.959 1630.73,648.351 1627.99,643.801 1622.88,635.642 1617.59,627.484 1612.11,619.326 \n",
       "  1610.56,617.093 1606.36,611.168 1600.38,603.01 1594.21,594.851 1590.39,589.951 1587.79,586.693 1581.1,578.535 1574.21,570.377 1570.22,565.779 1567.06,562.219 \n",
       "  1559.62,554.06 1551.98,545.902 1550.05,543.895 1544.01,537.744 1535.79,529.586 1529.88,523.87 1527.3,521.427 1518.47,513.269 1509.71,505.363 1509.43,505.111 \n",
       "  1499.96,496.953 1490.28,488.795 1489.54,488.187 1480.17,480.636 1469.81,472.478 1469.38,472.141 1458.99,464.32 1449.21,457.104 1447.9,456.162 1436.33,448.004 \n",
       "  1429.04,442.965 1424.41,439.845 1412.06,431.687 1408.87,429.618 1399.22,423.529 1388.7,417.005 1385.99,415.371 1372.21,407.213 1368.53,405.07 1357.89,399.054 \n",
       "  1348.36,393.756 1343.07,390.896 1328.19,383.002 1327.68,382.738 1311.57,374.58 1308.02,372.811 1294.8,366.421 1287.85,363.121 1277.31,358.263 1267.68,353.904 \n",
       "  1259.02,350.105 1247.52,345.137 1239.87,341.947 1227.35,336.799 1219.76,333.789 1207.18,328.871 1198.58,325.63 1187.01,321.334 1176.21,317.472 1166.84,314.171 \n",
       "  1152.51,309.314 1146.67,307.364 1127.31,301.156 1126.5,300.9 1106.33,294.787 1100.16,292.998 1086.16,288.996 1070.94,284.839 1065.99,283.508 1045.83,278.334 \n",
       "  1039.05,276.681 1025.66,273.46 1005.49,268.862 1003.92,268.523 985.318,264.57 965.149,260.531 964.263,260.365 944.98,256.79 924.811,253.296 918.048,252.207 \n",
       "  904.642,250.076 884.473,247.112 864.304,244.391 861.52,244.048 844.135,241.939 823.966,239.73 803.797,237.761 783.628,236.03 781.743,235.89 763.459,234.555 \n",
       "  743.29,233.318 723.121,232.316 702.952,231.55 682.783,231.02 662.614,230.725 642.445,230.666 622.276,230.843 602.107,231.256 581.938,231.904 561.769,232.787 \n",
       "  541.6,233.907 521.431,235.262 513.465,235.89 501.262,236.865 481.093,238.716 460.924,240.805 440.755,243.133 433.559,244.048 420.585,245.721 400.416,248.564 \n",
       "  380.247,251.648 376.864,252.207 360.078,255.012 339.909,258.629 330.843,260.365 319.74,262.52 299.571,266.682 291.157,268.523 279.402,271.13 259.233,275.854 \n",
       "  255.881,276.681 239.064,280.889 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#972765; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,239.491 254.468,235.89 259.233,234.79 279.402,230.372 292.13,227.732 299.571,226.208 319.74,222.311 334.809,219.574 339.909,218.659 360.078,215.27 \n",
       "  380.247,212.112 385.042,211.415 400.416,209.211 420.585,206.545 440.755,204.106 448.497,203.257 460.924,201.911 481.093,199.951 501.262,198.214 521.431,196.702 \n",
       "  541.6,195.413 547.557,195.099 561.769,194.358 581.938,193.528 602.107,192.92 622.276,192.532 642.445,192.366 662.614,192.421 682.783,192.698 702.952,193.196 \n",
       "  723.121,193.916 743.29,194.856 747.504,195.099 763.459,196.03 783.628,197.43 803.797,199.054 823.966,200.903 844.135,202.976 846.609,203.257 864.304,205.297 \n",
       "  884.473,207.849 904.642,210.628 909.923,211.415 924.811,213.662 944.98,216.936 960.167,219.574 965.149,220.45 985.318,224.231 1002.91,227.732 1005.49,228.251 \n",
       "  1025.66,232.552 1040.5,235.89 1045.83,237.104 1065.99,241.939 1074.38,244.048 1086.16,247.052 1105.48,252.207 1106.33,252.438 1126.5,258.138 1134.05,260.365 \n",
       "  1146.67,264.135 1160.77,268.523 1166.84,270.436 1185.88,276.681 1187.01,277.056 1207.18,284.019 1209.47,284.839 1227.35,291.328 1231.79,292.998 1247.52,298.993 \n",
       "  1253,301.156 1267.68,307.031 1273.21,309.314 1287.85,315.456 1292.51,317.472 1308.02,324.288 1310.99,325.63 1328.19,333.545 1328.71,333.789 1345.66,341.947 \n",
       "  1348.36,343.266 1361.96,350.105 1368.53,353.465 1377.66,358.263 1388.7,364.161 1392.82,366.421 1407.44,374.58 1408.87,375.393 1421.47,382.738 1429.04,387.221 \n",
       "  1435.08,390.896 1448.26,399.054 1449.21,399.649 1460.92,407.213 1469.38,412.774 1473.23,415.371 1485.11,423.529 1489.54,426.626 1496.61,431.687 1507.79,439.845 \n",
       "  1509.71,441.278 1518.54,448.004 1529.04,456.162 1529.88,456.826 1539.12,464.32 1548.98,472.478 1550.05,473.386 1558.44,480.636 1567.68,488.795 1570.22,491.09 \n",
       "  1576.57,496.953 1585.22,505.111 1590.39,510.098 1593.6,513.269 1601.69,521.427 1609.58,529.586 1610.56,530.625 1617.14,537.744 1624.49,545.902 1630.73,553.005 \n",
       "  1631.64,554.06 1638.47,562.219 1645.12,570.377 1650.9,577.656 1651.58,578.535 1657.74,586.693 1663.72,594.851 1669.52,603.01 1671.07,605.251 1675.06,611.168 \n",
       "  1680.4,619.326 1685.56,627.484 1690.54,635.642 1691.23,636.826 1695.26,643.801 1699.81,651.959 1704.17,660.117 1708.37,668.275 1711.4,674.433 1712.37,676.433 \n",
       "  1716.15,684.592 1719.75,692.75 1723.19,700.908 1726.46,709.066 1729.56,717.225 1731.57,722.842 1732.47,725.383 1735.18,733.541 1737.72,741.699 1740.09,749.857 \n",
       "  1742.3,758.016 1744.34,766.174 1746.21,774.332 1747.92,782.49 1749.46,790.648 1750.84,798.807 1751.74,804.925 1752.04,806.965 1753.06,815.123 1753.92,823.281 \n",
       "  1754.61,831.439 1755.15,839.598 1755.51,847.756 1755.72,855.914 1755.76,864.072 1755.64,872.231 1755.35,880.389 1754.9,888.547 1754.29,896.705 1753.51,904.863 \n",
       "  1752.57,913.022 1751.74,919.14 1751.46,921.18 1750.17,929.338 1748.71,937.496 1747.09,945.654 1745.3,953.813 1743.34,961.971 1741.21,970.129 1738.92,978.287 \n",
       "  1736.47,986.445 1733.84,994.604 1731.57,1001.24 1731.04,1002.76 1728.03,1010.92 1724.85,1019.08 1721.49,1027.24 1717.97,1035.39 1714.28,1043.55 1711.4,1049.63 \n",
       "  1710.4,1051.71 1706.29,1059.87 1702.01,1068.03 1697.56,1076.19 1692.93,1084.34 1691.23,1087.22 1688.07,1092.5 1683,1100.66 1677.75,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#a72d5f; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1741.98,1108.82 1746.94,1100.66 1751.73,1092.5 1751.74,1092.47 1756.27,1084.34 1760.64,1076.19 1764.85,1068.03 1768.9,1059.87 1771.91,1053.55 1772.77,1051.71 \n",
       "  1776.43,1043.55 1779.92,1035.39 1783.26,1027.24 1786.43,1019.08 1789.44,1010.92 1792.08,1003.38 1792.29,1002.76 1794.94,994.604 1797.42,986.445 1799.75,978.287 \n",
       "  1801.92,970.129 1803.93,961.971 1805.79,953.813 1807.49,945.654 1809.03,937.496 1810.41,929.338 1811.63,921.18 1812.25,916.445 1812.69,913.022 1813.58,904.863 \n",
       "  1814.32,896.705 1814.9,888.547 1815.33,880.389 1815.6,872.231 1815.71,864.072 1815.68,855.914 1815.48,847.756 1815.13,839.598 1814.63,831.439 1813.97,823.281 \n",
       "  1813.15,815.123 1812.25,807.51 1812.18,806.965 1811.04,798.807 1809.74,790.648 1808.28,782.49 1806.66,774.332 1804.88,766.174 1802.95,758.016 1800.86,749.857 \n",
       "  1798.61,741.699 1796.2,733.541 1793.63,725.383 1792.08,720.73 1790.89,717.225 1787.96,709.066 1784.86,700.908 1781.61,692.75 1778.19,684.592 1774.62,676.433 \n",
       "  1771.91,670.517 1770.87,668.275 1766.9,660.117 1762.77,651.959 1758.47,643.801 1754.02,635.642 1751.74,631.628 1749.35,627.484 1744.48,619.326 1739.44,611.168 \n",
       "  1734.23,603.01 1731.57,598.972 1728.81,594.851 1723.16,586.693 1717.35,578.535 1711.4,570.425 1711.37,570.377 1705.1,562.219 1698.65,554.06 1692.04,545.902 \n",
       "  1691.23,544.938 1685.13,537.744 1678.03,529.586 1671.07,521.773 1670.75,521.427 1663.15,513.269 1655.37,505.111 1650.9,500.523 1647.34,496.953 1639.04,488.795 \n",
       "  1630.73,480.806 1630.55,480.636 1621.7,472.478 1612.66,464.32 1610.56,462.463 1603.28,456.162 1593.67,448.004 1590.39,445.274 1583.73,439.845 1573.51,431.687 \n",
       "  1570.22,429.105 1562.95,423.529 1552.12,415.371 1550.05,413.845 1540.88,407.213 1529.88,399.406 1529.38,399.054 1517.41,390.896 1509.71,385.736 1505.13,382.738 \n",
       "  1492.46,374.58 1489.54,372.735 1479.33,366.421 1469.38,360.371 1465.82,358.263 1451.84,350.105 1449.21,348.591 1437.35,341.947 1429.04,337.366 1422.38,333.789 \n",
       "  1408.87,326.639 1406.91,325.63 1390.84,317.472 1388.7,316.404 1374.13,309.314 1368.53,306.63 1356.79,301.156 1348.36,297.284 1338.76,292.998 1328.19,288.348 \n",
       "  1319.98,284.839 1308.02,279.803 1300.38,276.681 1287.85,271.633 1279.89,268.523 1267.68,263.824 1258.4,260.365 1247.52,256.36 1235.83,252.207 1227.35,249.229 \n",
       "  1212.06,244.048 1207.18,242.416 1187.01,235.91 1186.95,235.89 1166.84,229.723 1160.09,227.732 1146.67,223.823 1131.48,219.574 1126.5,218.199 1106.33,212.858 \n",
       "  1100.64,211.415 1086.16,207.793 1067.19,203.257 1065.99,202.975 1045.83,198.438 1030.21,195.099 1025.66,194.137 1005.49,190.097 988.814,186.941 985.318,186.287 \n",
       "  965.149,182.734 944.98,179.399 940.993,178.783 924.811,176.312 904.642,173.449 884.473,170.802 883,170.624 864.304,168.397 844.135,166.208 823.966,164.233 \n",
       "  803.797,162.471 803.73,162.466 783.628,160.941 763.459,159.622 743.29,158.514 723.121,157.617 702.952,156.931 682.783,156.456 662.614,156.192 642.445,156.139 \n",
       "  622.276,156.297 602.107,156.667 581.938,157.247 561.769,158.039 541.6,159.041 521.431,160.255 501.262,161.68 491.565,162.466 481.093,163.326 460.924,165.194 \n",
       "  440.755,167.276 420.585,169.572 412.126,170.624 400.416,172.098 380.247,174.854 360.078,177.825 354.017,178.783 339.909,181.039 319.74,184.483 306.207,186.941 \n",
       "  299.571,188.161 279.402,192.089 264.775,195.099 259.233,196.253 239.064,200.679 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#ba3655; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,164.019 246.492,162.466 259.233,159.833 279.402,155.875 287.838,154.308 299.571,152.153 319.74,148.658 335.137,146.15 339.909,145.381 360.078,142.338 \n",
       "  380.247,139.502 391.827,137.992 400.416,136.884 420.585,134.487 440.755,132.294 460.924,130.306 466.26,129.833 481.093,128.535 501.262,126.972 521.431,125.61 \n",
       "  541.6,124.451 561.769,123.492 581.938,122.736 602.107,122.181 622.276,121.828 642.445,121.677 662.614,121.727 682.783,121.979 702.952,122.433 723.121,123.089 \n",
       "  743.29,123.946 763.459,125.005 783.628,126.266 803.797,127.728 823.966,129.393 828.73,129.833 844.135,131.275 864.304,133.365 884.473,135.66 903.292,137.992 \n",
       "  904.642,138.161 924.811,140.894 944.98,143.834 959.829,146.15 965.149,146.989 985.318,150.38 1005.49,153.979 1007.23,154.308 1025.66,157.827 1045.83,161.891 \n",
       "  1048.54,162.466 1065.99,166.208 1085.62,170.624 1086.16,170.748 1106.33,175.556 1119.29,178.783 1126.5,180.601 1146.67,185.904 1150.46,186.941 1166.84,191.48 \n",
       "  1179.4,195.099 1187.01,197.318 1206.63,203.257 1207.18,203.425 1227.35,209.834 1232.15,211.415 1247.52,216.533 1256.34,219.574 1267.68,223.532 1279.33,227.732 \n",
       "  1287.85,230.843 1301.25,235.89 1308.02,238.476 1322.18,244.048 1328.19,246.446 1342.21,252.207 1348.36,254.767 1361.43,260.365 1368.53,263.451 1379.88,268.523 \n",
       "  1388.7,272.515 1397.65,276.681 1408.87,281.974 1414.78,284.839 1429.04,291.846 1431.32,292.998 1447.27,301.156 1449.21,302.162 1462.64,309.314 1469.38,312.952 \n",
       "  1477.54,317.472 1489.54,324.219 1492,325.63 1505.95,333.789 1509.71,336.02 1519.47,341.947 1529.88,348.375 1532.62,350.105 1545.34,358.263 1550.05,361.337 \n",
       "  1557.68,366.421 1569.71,374.58 1570.22,374.931 1581.3,382.738 1590.39,389.252 1592.63,390.896 1603.59,399.054 1610.56,404.341 1614.27,407.213 1624.62,415.371 \n",
       "  1630.73,420.277 1634.7,423.529 1644.47,431.687 1650.9,437.156 1653.99,439.845 1663.21,448.004 1671.07,455.092 1672.23,456.162 1680.91,464.32 1689.42,472.478 \n",
       "  1691.23,474.252 1697.63,480.636 1705.63,488.795 1711.4,494.811 1713.42,496.953 1720.93,505.111 1728.27,513.269 1731.57,517.025 1735.37,521.427 1742.24,529.586 \n",
       "  1748.95,537.744 1751.74,541.227 1755.42,545.902 1761.68,554.06 1767.77,562.219 1771.91,567.913 1773.67,570.377 1779.33,578.535 1784.84,586.693 1790.18,594.851 \n",
       "  1792.08,597.842 1795.3,603.01 1800.24,611.168 1805.01,619.326 1809.63,627.484 1812.25,632.272 1814.06,635.642 1818.29,643.801 1822.36,651.959 1826.28,660.117 \n",
       "  1830.04,668.275 1832.42,673.649 1833.63,676.433 1837.02,684.592 1840.26,692.75 1843.35,700.908 1846.29,709.066 1849.07,717.225 1851.7,725.383 1852.59,728.285 \n",
       "  1854.16,733.541 1856.44,741.699 1858.58,749.857 1860.57,758.016 1862.41,766.174 1864.1,774.332 1865.63,782.49 1867.02,790.648 1868.26,798.807 1869.35,806.965 \n",
       "  1870.28,815.123 1871.07,823.281 1871.71,831.439 1872.2,839.598 1872.53,847.756 1872.72,855.914 1872.76,863.444 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#ba3655; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,864.282 1872.65,872.231 1872.38,880.389 1871.97,888.547 1871.41,896.705 1870.7,904.863 1869.83,913.022 1868.82,921.18 1867.66,929.338 1866.35,937.496 \n",
       "  1864.88,945.654 1863.27,953.813 1861.51,961.971 1859.6,970.129 1857.53,978.287 1855.32,986.445 1852.96,994.604 1852.59,995.806 1850.41,1002.76 1847.7,1010.92 \n",
       "  1844.84,1019.08 1841.83,1027.24 1838.66,1035.39 1835.34,1043.55 1832.42,1050.43 1831.86,1051.71 1828.18,1059.87 1824.34,1068.03 1820.34,1076.19 1816.19,1084.34 \n",
       "  1812.25,1091.82 1811.88,1092.5 1807.34,1100.66 1802.65,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#cb4148; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1860.28,1108.82 1864.75,1100.66 1869.06,1092.5 1872.76,1085.26 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#cb4148; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,638.828 1871.16,635.642 1866.92,627.484 1862.53,619.326 1858,611.168 1853.31,603.01 1852.59,601.793 1848.4,594.851 1843.33,586.693 1838.1,578.535 \n",
       "  1832.73,570.377 1832.42,569.921 1827.11,562.219 1821.33,554.06 1815.39,545.902 1812.25,541.691 1809.25,537.744 1802.9,529.586 1796.38,521.427 1792.08,516.163 \n",
       "  1789.67,513.269 1782.72,505.111 1775.61,496.953 1771.91,492.799 1768.28,488.795 1760.71,480.636 1752.98,472.478 1751.74,471.197 1744.97,464.32 1736.76,456.162 \n",
       "  1731.57,451.106 1728.33,448.004 1719.63,439.845 1711.4,432.278 1710.75,431.687 1701.54,423.529 1692.15,415.371 1691.23,414.586 1682.43,407.213 1672.51,399.054 \n",
       "  1671.07,397.889 1662.24,390.896 1651.76,382.738 1650.9,382.075 1640.91,374.58 1630.73,367.069 1629.83,366.421 1618.36,358.263 1610.56,352.806 1606.62,350.105 \n",
       "  1594.52,341.947 1590.39,339.207 1582.05,333.789 1570.22,326.22 1569.28,325.63 1556.05,317.472 1550.05,313.832 1542.45,309.314 1529.88,301.962 1528.47,301.156 \n",
       "  1514,292.998 1509.71,290.615 1499.07,284.839 1489.54,279.738 1483.69,276.681 1469.38,269.302 1467.83,268.523 1451.38,260.365 1449.21,259.302 1434.33,252.207 \n",
       "  1429.04,249.712 1416.69,244.048 1408.87,240.506 1398.4,235.89 1388.7,231.667 1379.41,227.732 1368.53,223.183 1359.65,219.574 1348.36,215.04 1339.07,211.415 \n",
       "  1328.19,207.225 1317.58,203.257 1308.02,199.726 1295.11,195.099 1287.85,192.532 1271.55,186.941 1267.68,185.631 1247.52,179.016 1246.78,178.783 1227.35,172.692 \n",
       "  1220.51,170.624 1207.18,166.635 1192.73,162.466 1187.01,160.835 1166.84,155.294 1163.11,154.308 1146.67,150.015 1131.27,146.15 1126.5,144.968 1106.33,140.172 \n",
       "  1096.75,137.992 1086.16,135.609 1065.99,131.274 1058.96,129.833 1045.83,127.174 1025.66,123.29 1016.81,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#d84c3d; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1169.91,121.675 1187.01,126.165 1200.47,129.833 1207.18,131.682 1227.35,137.445 1229.2,137.992 1247.52,143.472 1256.17,146.15 1267.68,149.754 1281.77,154.308 \n",
       "  1287.85,156.297 1306.13,162.466 1308.02,163.112 1328.19,170.212 1329.33,170.624 1348.36,177.608 1351.47,178.783 1368.53,185.303 1372.7,186.941 1388.7,193.306 \n",
       "  1393.08,195.099 1408.87,201.63 1412.7,203.257 1429.04,210.287 1431.59,211.415 1449.21,219.29 1449.83,219.574 1467.4,227.732 1469.38,228.662 1484.37,235.89 \n",
       "  1489.54,238.416 1500.81,244.048 1509.71,248.563 1516.74,252.207 1529.88,259.118 1532.2,260.365 1547.16,268.523 1550.05,270.121 1561.66,276.681 1570.22,281.591 \n",
       "  1575.77,284.839 1589.49,292.998 1590.39,293.538 1602.76,301.156 1610.56,306.03 1615.71,309.314 1628.3,317.472 1630.73,319.066 1640.51,325.63 1650.9,332.707 \n",
       "  1652.45,333.789 1664.01,341.947 1671.07,347.007 1675.3,350.105 1686.28,358.263 1691.23,362.005 1696.97,366.421 1707.39,374.58 1711.4,377.773 1717.52,382.738 \n",
       "  1727.41,390.896 1731.57,394.39 1737.03,399.054 1746.4,407.213 1751.74,411.95 1755.53,415.371 1764.41,423.529 1771.91,430.557 1773.1,431.687 1781.49,439.845 \n",
       "  1789.73,448.004 1792.08,450.381 1797.7,456.162 1805.48,464.32 1812.25,471.574 1813.08,472.478 1820.41,480.636 1827.59,488.795 1832.42,494.408 1834.57,496.953 \n",
       "  1841.32,505.111 1847.92,513.269 1852.59,519.179 1854.33,521.427 1860.52,529.586 1866.56,537.744 1872.45,545.902 1872.76,546.34 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#e55c2f; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,475.407 1870.21,472.478 1862.97,464.32 1855.58,456.162 1852.59,452.917 1847.97,448.004 1840.15,439.845 1832.42,431.929 1832.18,431.687 1823.92,423.529 \n",
       "  1815.5,415.371 1812.25,412.279 1806.83,407.213 1797.95,399.054 1792.08,393.754 1788.86,390.896 1779.5,382.738 1771.91,376.237 1769.94,374.58 1760.09,366.421 \n",
       "  1751.74,359.627 1750.04,358.263 1739.66,350.105 1731.57,343.841 1729.08,341.947 1718.18,333.789 1711.4,328.8 1707.02,325.63 1695.56,317.472 1691.23,314.44 \n",
       "  1683.78,309.314 1671.75,301.156 1671.07,300.701 1659.3,292.998 1650.9,287.569 1646.58,284.839 1633.51,276.681 1630.73,274.971 1620.03,268.523 1610.56,262.891 \n",
       "  1606.22,260.365 1592.02,252.207 1590.39,251.284 1577.35,244.048 1570.22,240.147 1562.27,235.89 1550.05,229.428 1546.77,227.732 1530.79,219.574 1529.88,219.117 \n",
       "  1514.25,211.415 1509.71,209.21 1497.19,203.257 1489.54,199.67 1479.57,195.099 1469.38,190.484 1461.35,186.941 1449.21,181.639 1442.49,178.783 1429.04,173.124 \n",
       "  1422.94,170.624 1408.87,164.927 1402.63,162.466 1388.7,157.035 1381.51,154.308 1368.53,149.44 1359.51,146.15 1348.36,142.131 1336.54,137.992 1328.19,135.099 \n",
       "  1312.53,129.833 1308.02,128.333 1287.85,121.828 1287.37,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#ef6d21; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1386.33,121.675 1388.7,122.534 1408.29,129.833 1408.87,130.05 1429.04,137.852 1429.39,137.992 1449.21,145.948 1449.7,146.15 1469.29,154.308 1469.38,154.344 \n",
       "  1488.19,162.466 1489.54,163.058 1506.46,170.624 1509.71,172.098 1524.14,178.783 1529.88,181.475 1541.27,186.941 1550.05,191.203 1557.9,195.099 1570.22,201.294 \n",
       "  1574.04,203.257 1589.72,211.415 1590.39,211.766 1604.9,219.574 1610.56,222.659 1619.67,227.732 1630.73,233.964 1634.07,235.89 1648.07,244.048 1650.9,245.72 \n",
       "  1661.66,252.207 1671.07,257.953 1674.94,260.365 1687.85,268.523 1691.23,270.687 1700.42,276.681 1711.4,283.954 1712.72,284.839 1724.64,292.998 1731.57,297.809 \n",
       "  1736.31,301.156 1747.68,309.314 1751.74,312.274 1758.75,317.472 1769.59,325.63 1771.91,327.405 1780.11,333.789 1790.44,341.947 1792.08,343.265 1800.45,350.105 \n",
       "  1810.28,358.263 1812.25,359.924 1819.82,366.421 1829.17,374.58 1832.42,377.465 1838.26,382.738 1847.14,390.896 1852.59,395.98 1855.82,399.054 1864.26,407.213 \n",
       "  1872.55,415.371 1872.76,415.577 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#f68211; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,362.896 1867.54,358.263 1858.2,350.105 1852.59,345.279 1848.65,341.947 1838.84,333.789 1832.42,328.521 1828.83,325.63 1818.55,317.472 1812.25,312.545 \n",
       "  1808.04,309.314 1797.27,301.156 1792.08,297.283 1786.23,292.998 1774.95,284.839 1771.91,282.676 1763.35,276.681 1751.74,268.671 1751.52,268.523 1739.32,260.365 \n",
       "  1731.57,255.256 1726.86,252.207 1714.09,244.048 1711.4,242.355 1700.95,235.89 1691.23,229.958 1687.52,227.732 1673.72,219.574 1671.07,218.026 1659.52,211.415 \n",
       "  1650.9,206.544 1644.96,203.257 1630.73,195.468 1630.04,195.099 1614.63,186.941 1610.56,184.81 1598.79,178.783 1590.39,174.528 1582.51,170.624 1570.22,164.606 \n",
       "  1565.75,162.466 1550.05,155.03 1548.49,154.308 1530.67,146.15 1529.88,145.793 1512.25,137.992 1509.71,136.883 1493.21,129.833 1489.54,128.282 1473.53,121.675 \n",
       "  \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#fa9607; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1552.39,121.675 1570.22,129.745 1570.41,129.833 1587.85,137.992 1590.39,139.191 1604.79,146.15 1610.56,148.97 1621.25,154.308 1630.73,159.093 1637.27,162.466 \n",
       "  1650.9,169.571 1652.88,170.624 1668.03,178.783 1671.07,180.436 1682.77,186.941 1691.23,191.701 1697.16,195.099 1711.22,203.257 1711.4,203.368 1724.84,211.415 \n",
       "  1731.57,215.499 1738.17,219.574 1751.2,227.732 1751.74,228.072 1763.85,235.89 1771.91,241.161 1776.25,244.048 1788.34,252.207 1792.08,254.765 1800.13,260.365 \n",
       "  1811.69,268.523 1812.25,268.923 1822.91,276.681 1832.42,283.698 1833.94,284.839 1844.66,292.998 1852.59,299.123 1855.18,301.156 1865.41,309.314 1872.76,315.252 \n",
       "  \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#fbac11; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,271.443 1868.88,268.523 1857.89,260.365 1852.59,256.481 1846.65,252.207 1835.17,244.048 1832.42,242.116 1823.39,235.89 1812.25,228.308 1811.39,227.732 \n",
       "  1799.03,219.574 1792.08,215.039 1786.42,211.415 1773.53,203.257 1771.91,202.246 1760.27,195.099 1751.74,189.93 1746.72,186.941 1732.85,178.783 1731.57,178.039 \n",
       "  1718.58,170.624 1711.4,166.581 1703.97,162.466 1691.23,155.505 1689,154.308 1673.62,146.15 1671.07,144.813 1657.79,137.992 1650.9,134.486 1641.56,129.833 \n",
       "  1630.73,124.5 1624.87,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#f9c32a; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1692.39,121.675 1708,129.833 1711.4,131.63 1723.22,137.992 1731.57,142.543 1738.07,146.15 1751.74,153.821 1752.59,154.308 1766.7,162.466 1771.91,165.513 \n",
       "  1780.49,170.624 1792.08,177.607 1794,178.783 1807.14,186.941 1812.25,190.151 1819.99,195.099 1832.42,203.142 1832.59,203.257 1844.83,211.415 1852.59,216.647 \n",
       "  1856.85,219.574 1868.59,227.732 1872.76,230.664 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#f5d949; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,192.364 1864.55,186.941 1852.59,179.124 1852.06,178.783 1839.2,170.624 1832.42,166.367 1826.09,162.466 1812.71,154.308 1812.25,154.03 1798.94,146.15 \n",
       "  1792.08,142.13 1784.89,137.992 1771.91,130.61 1770.52,129.833 1755.76,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#f2ee74; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1815.71,121.675 1829.71,129.833 1832.42,131.426 1843.37,137.992 1852.59,143.574 1856.77,146.15 1869.85,154.308 1872.76,156.137 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#cb4148; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  278.066,121.675 259.233,125.207 239.064,129.191 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0302)\" style=\"stroke:#060419; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  570.191,643.801 561.769,644.887 541.6,648.181 522.492,651.959 521.431,652.177 501.262,657.044 490.169,660.117 481.093,662.735 464.083,668.275 460.924,669.348 \n",
       "  442.206,676.433 440.755,677.008 423.364,684.592 420.585,685.86 406.782,692.75 400.416,696.084 391.927,700.908 380.247,707.891 378.424,709.066 366.428,717.225 \n",
       "  360.078,721.793 355.428,725.383 345.47,733.541 339.909,738.396 336.366,741.699 328.188,749.857 320.585,758.016 319.74,758.995 313.924,766.174 307.854,774.332 \n",
       "  302.323,782.49 299.571,786.988 297.458,790.648 293.257,798.807 289.565,806.965 286.383,815.123 283.709,823.281 281.545,831.439 279.89,839.598 279.402,843.074 \n",
       "  278.78,847.756 278.177,855.914 278.057,864.072 278.418,872.231 279.262,880.389 279.402,881.252 280.654,888.547 282.564,896.705 284.982,904.863 287.91,913.022 \n",
       "  291.347,921.18 295.294,929.338 299.571,937.17 299.76,937.496 305.021,945.654 310.822,953.813 317.162,961.971 319.74,965.028 324.315,970.129 332.205,978.287 \n",
       "  339.909,985.713 340.721,986.445 350.372,994.604 360.078,1002.32 360.677,1002.76 372.344,1010.92 380.247,1016.15 385.016,1019.08 399.016,1027.24 400.416,1028.01 \n",
       "  414.858,1035.39 420.585,1038.19 432.612,1043.55 440.755,1047.03 452.866,1051.71 460.924,1054.7 476.478,1059.87 481.093,1061.34 501.262,1067.04 505.263,1068.03 \n",
       "  521.431,1071.85 541.6,1075.92 543.206,1076.19 561.769,1079.16 581.938,1081.71 602.107,1083.58 614.965,1084.34 622.276,1084.76 642.445,1085.25 662.614,1085.09 \n",
       "  680.917,1084.34 682.783,1084.27 702.952,1082.73 723.121,1080.52 743.29,1077.63 751.441,1076.19 763.459,1073.97 783.628,1069.56 789.64,1068.03 803.797,1064.28 \n",
       "  818.47,1059.87 823.966,1058.15 842.271,1051.71 844.135,1051.03 862.513,1043.55 864.304,1042.79 880.146,1035.39 884.473,1033.28 895.809,1027.24 904.642,1022.29 \n",
       "  909.948,1019.08 922.738,1010.92 924.811,1009.52 934.142,1002.76 944.766,994.604 944.98,994.428 954.114,986.445 962.856,978.287 965.149,975.992 970.645,970.129 \n",
       "  977.737,961.971 984.272,953.813 985.318,952.385 989.965,945.654 995.074,937.496 999.658,929.338 1003.72,921.18 1005.49,917.1 1007.16,913.022 1010.01,904.863 \n",
       "  1012.36,896.705 1014.22,888.547 1015.58,880.389 1016.44,872.231 1016.82,864.072 1016.69,855.914 1016.07,847.756 1014.96,839.598 1013.35,831.439 1011.24,823.281 \n",
       "  1008.64,815.123 1005.55,806.965 1005.49,806.824 1001.75,798.807 997.432,790.648 992.585,782.49 987.214,774.332 985.318,771.708 981.074,766.174 974.26,758.016 \n",
       "  966.891,749.857 965.149,748.065 958.559,741.699 949.521,733.541 944.98,729.695 939.533,725.383 928.592,717.225 924.811,714.56 916.428,709.066 904.642,701.744 \n",
       "  903.187,700.908 888.247,692.75 884.473,690.786 871.51,684.592 864.304,681.303 852.593,676.433 844.135,673.068 830.79,668.275 823.966,665.926 805.05,660.117 \n",
       "  803.797,659.748 783.628,654.52 772.163,651.959 763.459,650.088 743.29,646.447 725.181,643.801 723.121,643.511 702.952,641.339 682.783,639.836 662.614,639.001 \n",
       "  642.445,638.834 622.276,639.335 602.107,640.504 581.938,642.342 570.191,643.801 \n",
       "  \"/>\n",
       "<circle clip-path=\"url(#clip0302)\" cx=\"647.487\" cy=\"862.033\" r=\"14\" fill=\"#e26f46\" fill-rule=\"evenodd\" fill-opacity=\"1\" stroke=\"#000000\" stroke-opacity=\"1\" stroke-width=\"2.56\"/>\n",
       "<path clip-path=\"url(#clip0300)\" d=\"\n",
       "M1438.61 297.355 L1815.16 297.355 L1815.16 176.395 L1438.61 176.395  Z\n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip0300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1438.61,297.355 1815.16,297.355 1815.16,176.395 1438.61,176.395 1438.61,297.355 \n",
       "  \"/>\n",
       "<circle clip-path=\"url(#clip0300)\" cx=\"1525.01\" cy=\"236.875\" r=\"21\" fill=\"#e26f46\" fill-rule=\"evenodd\" fill-opacity=\"1\" stroke=\"#000000\" stroke-opacity=\"1\" stroke-width=\"2.56\"/>\n",
       "<g clip-path=\"url(#clip0300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 1592.21, 254.375)\" x=\"1592.21\" y=\"254.375\">optimum</text>\n",
       "</g>\n",
       "</svg>\n"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "contour( x,y,loss2d',       #notice: loss2d'\n",
    "         xlims = (1,5),\n",
    "         ylims = (-1,0),\n",
    "         legend = false,\n",
    "         levels = 21,\n",
    "         title = \"Contour plot of loss function\",\n",
    "         xlabel = \"x\",\n",
    "         ylabel = \"y\" )\n",
    "\n",
    "scatter!([2],[-0.75],label=\"optimum\",legend=true)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "minimum at (x,y)=      2.000    -0.750\n"
     ]
    }
   ],
   "source": [
    "Sol = optimize(fn2,[0.0;0.0])        #use p->lossfn(p,other arguments) if \n",
    "                                     #there are additional (non-choice) arguments\n",
    "printlnPs(\"minimum at (x,y)= \",Optim.minimizer(Sol))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Several Choice Variables: Bounds on the Solutions\n",
    "\n",
    "The next few cells discuss how to impose bounds on the solution.\n",
    "\n",
    "In the example below, we impose $2.75 \\leq x$ (a lower bound) and $y \\leq -0.3$ (an upper bound). We will see that only one of these restrictions binds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"480\" height=\"320\" viewBox=\"0 0 1920 1280\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip0700\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"1920\" height=\"1280\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<path clip-path=\"url(#clip0700)\" d=\"\n",
       "M0 1280 L1920 1280 L1920 0 L0 0  Z\n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip0701\">\n",
       "    <rect x=\"384\" y=\"0\" width=\"1345\" height=\"1280\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<path clip-path=\"url(#clip0700)\" d=\"\n",
       "M239.064 1108.82 L1872.76 1108.82 L1872.76 121.675 L239.064 121.675  Z\n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip0702\">\n",
       "    <rect x=\"239\" y=\"121\" width=\"1635\" height=\"988\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,1108.82 239.064,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  647.487,1108.82 647.487,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1055.91,1108.82 1055.91,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1464.33,1108.82 1464.33,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1872.76,1108.82 1872.76,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,1108.82 1872.76,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,911.39 1872.76,911.39 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,713.961 1872.76,713.961 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,516.533 1872.76,516.533 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,319.104 1872.76,319.104 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  239.064,121.675 1872.76,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,1108.82 1872.76,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,1108.82 239.064,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,1108.82 239.064,1096.97 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  647.487,1108.82 647.487,1096.97 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1055.91,1108.82 1055.91,1096.97 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1464.33,1108.82 1464.33,1096.97 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,1108.82 1872.76,1096.97 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,1108.82 258.668,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,911.39 258.668,911.39 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,713.961 258.668,713.961 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,516.533 258.668,516.533 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,319.104 258.668,319.104 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,121.675 258.668,121.675 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 239.064, 1160.42)\" x=\"239.064\" y=\"1160.42\">1</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 647.487, 1160.42)\" x=\"647.487\" y=\"1160.42\">2</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1055.91, 1160.42)\" x=\"1055.91\" y=\"1160.42\">3</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1464.33, 1160.42)\" x=\"1464.33\" y=\"1160.42\">4</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1872.76, 1160.42)\" x=\"1872.76\" y=\"1160.42\">5</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 1126.32)\" x=\"219.864\" y=\"1126.32\">-1.0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 928.89)\" x=\"219.864\" y=\"928.89\">-0.8</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 731.461)\" x=\"219.864\" y=\"731.461\">-0.6</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 534.033)\" x=\"219.864\" y=\"534.033\">-0.4</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 336.604)\" x=\"219.864\" y=\"336.604\">-0.2</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 219.864, 139.175)\" x=\"219.864\" y=\"139.175\">0.0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:84px; text-anchor:middle;\" transform=\"rotate(0, 1055.91, 73.2)\" x=\"1055.91\" y=\"73.2\">Contour plot of loss function</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:66px; text-anchor:middle;\" transform=\"rotate(0, 1055.91, 1237.88)\" x=\"1055.91\" y=\"1237.88\">x</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:66px; text-anchor:middle;\" transform=\"rotate(-90, 94.0861, 615.247)\" x=\"94.0861\" y=\"615.247\">y</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#130a34; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  321.681,1108.82 319.74,1107.89 305.512,1100.66 299.571,1097.54 290.538,1092.5 279.402,1086.07 276.568,1084.34 263.673,1076.19 259.233,1073.27 251.664,1068.03 \n",
       "  240.338,1059.87 239.064,1058.91 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#130a34; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  973.198,1108.82 985.318,1102.84 989.488,1100.66 1004.55,1092.5 1005.49,1091.98 1018.35,1084.34 1025.66,1079.84 1031.29,1076.19 1043.38,1068.03 1045.83,1066.31 \n",
       "  1054.54,1059.87 1065.14,1051.71 1065.99,1051.03 1074.86,1043.55 1084.11,1035.39 1086.16,1033.5 1092.63,1027.24 1100.65,1019.08 1106.33,1013 1108.19,1010.92 \n",
       "  1115.09,1002.76 1121.6,994.604 1126.5,988.072 1127.67,986.445 1133.17,978.287 1138.29,970.129 1143.05,961.971 1146.67,955.219 1147.4,953.813 1151.24,945.654 \n",
       "  1154.73,937.496 1157.87,929.338 1160.64,921.18 1163.06,913.022 1165.11,904.863 1166.81,896.705 1166.84,896.552 1168.11,888.547 1169.05,880.389 1169.66,872.231 \n",
       "  1169.91,864.072 1169.83,855.914 1169.4,847.756 1168.62,839.598 1167.5,831.439 1166.84,827.735 1166.01,823.281 1164.13,815.123 1161.89,806.965 1159.3,798.807 \n",
       "  1156.34,790.648 1153.03,782.49 1149.36,774.332 1146.67,768.875 1145.28,766.174 1140.72,758.016 1135.78,749.857 1130.47,741.699 1126.5,736.007 1124.71,733.541 \n",
       "  1118.39,725.383 1111.69,717.225 1106.33,711.068 1104.51,709.066 1096.69,700.908 1088.47,692.75 1086.16,690.572 1079.54,684.592 1070.08,676.433 1065.99,673.067 \n",
       "  1059.9,668.275 1049.08,660.117 1045.83,657.764 1037.39,651.959 1025.66,644.193 1025.03,643.801 1011.54,635.642 1005.49,632.115 997.085,627.484 985.318,621.22 \n",
       "  981.541,619.326 965.149,611.378 964.687,611.168 946.166,603.01 944.98,602.503 925.794,594.851 924.811,594.471 904.642,587.216 903.069,586.693 884.473,580.69 \n",
       "  877.205,578.535 864.304,574.818 847.381,570.377 844.135,569.548 823.966,564.903 810.899,562.219 803.797,560.799 783.628,557.255 763.459,554.201 762.354,554.06 \n",
       "  743.29,551.698 723.121,549.676 702.952,548.129 682.783,547.058 662.614,546.463 642.445,546.344 622.276,546.701 602.107,547.534 581.938,548.843 561.769,550.628 \n",
       "  541.6,552.888 532.962,554.06 521.431,555.667 501.262,558.966 483.942,562.219 481.093,562.768 460.924,567.163 447.685,570.377 440.755,572.107 420.585,577.659 \n",
       "  417.674,578.535 400.416,583.879 392.041,586.693 380.247,590.775 369.328,594.851 360.078,598.411 348.939,603.01 339.909,606.857 330.433,611.168 319.74,616.193 \n",
       "  313.471,619.326 299.571,626.511 297.794,627.484 283.409,635.642 279.402,637.998 270.06,643.801 259.233,650.781 257.499,651.959 245.944,660.117 239.064,665.175 \n",
       "  \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#240b4e; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,564.4 243.5,562.219 259.233,554.689 260.617,554.06 279.055,545.902 279.402,545.753 299.022,537.744 299.571,537.525 319.74,529.949 320.769,529.586 \n",
       "  339.909,522.989 344.748,521.427 360.078,516.597 371.405,513.269 380.247,510.732 400.416,505.365 401.452,505.111 420.585,500.525 436.876,496.953 440.755,496.121 \n",
       "  460.924,492.199 480.425,488.795 481.093,488.681 501.262,485.63 521.431,482.973 541.6,480.71 542.391,480.636 561.769,478.878 581.938,477.433 602.107,476.374 \n",
       "  622.276,475.699 642.445,475.41 662.614,475.507 682.783,475.988 702.952,476.855 723.121,478.107 743.29,479.745 752.178,480.636 763.459,481.792 783.628,484.252 \n",
       "  803.797,487.106 814.284,488.795 823.966,490.388 844.135,494.11 858.041,496.953 864.304,498.262 884.473,502.891 893.358,505.111 904.642,507.996 923.712,513.269 \n",
       "  924.811,513.58 944.98,519.721 950.219,521.427 965.149,526.41 974.081,529.586 985.318,533.681 995.822,537.744 1005.49,541.578 1015.79,545.902 1025.66,550.151 \n",
       "  1034.26,554.06 1045.83,559.454 1051.46,562.219 1065.99,569.548 1067.56,570.377 1082.55,578.535 1086.16,580.556 1096.63,586.693 1106.33,592.554 1109.97,594.851 \n",
       "  1122.51,603.01 1126.5,605.691 1134.32,611.168 1145.6,619.326 1146.67,620.131 1156.11,627.484 1166.22,635.642 1166.84,636.161 1175.63,643.801 1184.66,651.959 \n",
       "  1187.01,654.159 1193.13,660.117 1201.17,668.275 1207.18,674.633 1208.82,676.433 1215.94,684.592 1222.74,692.75 1227.35,698.554 1229.15,700.908 1235.1,709.066 \n",
       "  1240.74,717.225 1246.07,725.383 1247.52,727.732 1250.97,733.541 1255.53,741.699 1259.79,749.857 1263.75,758.016 1267.41,766.174 1267.68,766.851 1270.67,774.332 \n",
       "  1273.63,782.49 1276.31,790.648 1278.69,798.807 1280.79,806.965 1282.6,815.123 1284.12,823.281 1285.35,831.439 1286.29,839.598 1286.94,847.756 1287.3,855.914 \n",
       "  1287.37,864.072 1287.15,872.231 1286.65,880.389 1285.85,888.547 1284.77,896.705 1283.39,904.863 1281.73,913.022 1279.78,921.18 1277.54,929.338 1275.01,937.496 \n",
       "  1272.19,945.654 1269.08,953.813 1267.68,957.157 1265.61,961.971 1261.8,970.129 1257.7,978.287 1253.29,986.445 1248.58,994.604 1247.52,996.342 1243.44,1002.76 \n",
       "  1237.96,1010.92 1232.16,1019.08 1227.35,1025.52 1226.02,1027.24 1219.38,1035.39 1212.42,1043.55 1207.18,1049.43 1205.07,1051.71 1197.19,1059.87 1188.98,1068.03 \n",
       "  1187.01,1069.91 1180.19,1076.19 1170.98,1084.34 1166.84,1087.88 1161.21,1092.5 1150.92,1100.66 1146.67,1103.91 1140,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#390962; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1263.17,1108.82 1267.68,1104.68 1271.92,1100.66 1280.24,1092.5 1287.85,1084.76 1288.25,1084.34 1295.75,1076.19 1302.97,1068.03 1308.02,1062.08 1309.85,1059.87 \n",
       "  1316.3,1051.71 1322.49,1043.55 1328.19,1035.68 1328.39,1035.39 1333.87,1027.24 1339.08,1019.08 1344.03,1010.92 1348.36,1003.37 1348.7,1002.76 1353,994.604 \n",
       "  1357.03,986.445 1360.82,978.287 1364.34,970.129 1367.61,961.971 1368.53,959.482 1370.57,953.813 1373.25,945.654 1375.68,937.496 1377.86,929.338 1379.79,921.18 \n",
       "  1381.47,913.022 1382.91,904.863 1384.09,896.705 1385.03,888.547 1385.71,880.389 1386.15,872.231 1386.34,864.072 1386.27,855.914 1385.96,847.756 1385.4,839.598 \n",
       "  1384.59,831.439 1383.53,823.281 1382.22,815.123 1380.66,806.965 1378.86,798.807 1376.8,790.648 1374.49,782.49 1371.94,774.332 1369.13,766.174 1368.53,764.565 \n",
       "  1366.01,758.016 1362.61,749.857 1358.96,741.699 1355.05,733.541 1350.88,725.383 1348.36,720.735 1346.4,717.225 1341.59,709.066 1336.51,700.908 1331.16,692.75 \n",
       "  1328.19,688.428 1325.48,684.592 1319.43,676.433 1313.11,668.275 1308.02,661.983 1306.47,660.117 1299.39,651.959 1292.04,643.801 1287.85,639.334 1284.29,635.642 \n",
       "  1276.12,627.484 1267.68,619.352 1267.66,619.326 1258.62,611.168 1249.28,603.01 1247.52,601.515 1239.38,594.851 1229.1,586.693 1227.35,585.34 1218.21,578.535 \n",
       "  1207.18,570.557 1206.92,570.377 1194.9,562.219 1187.01,557.01 1182.37,554.06 1169.2,545.902 1166.84,544.477 1155.24,537.744 1146.67,532.889 1140.6,529.586 \n",
       "  1126.5,522.106 1125.17,521.427 1108.74,513.269 1106.33,512.1 1091.28,505.111 1086.16,502.788 1072.7,496.953 1065.99,494.109 1052.84,488.795 1045.83,486.023 \n",
       "  1031.49,480.636 1025.66,478.492 1008.41,472.478 1005.49,471.482 985.318,464.973 983.169,464.32 965.149,458.953 955.162,456.162 944.98,453.373 924.811,448.21 \n",
       "  923.944,448.004 904.642,443.499 887.706,439.845 884.473,439.161 864.304,435.241 844.232,431.687 844.135,431.67 823.966,428.508 803.797,425.687 786.233,423.529 \n",
       "  783.628,423.215 763.459,421.117 743.29,419.356 723.121,417.929 702.952,416.839 682.783,416.084 662.614,415.664 642.445,415.58 622.276,415.832 602.107,416.419 \n",
       "  581.938,417.342 561.769,418.6 541.6,420.194 521.431,422.124 508.922,423.529 501.262,424.405 481.093,427.055 460.924,430.046 450.996,431.687 440.755,433.412 \n",
       "  420.585,437.158 407.345,439.845 400.416,441.279 380.247,445.808 371.182,448.004 360.078,450.746 339.909,456.09 339.655,456.162 319.74,461.91 311.891,464.32 \n",
       "  299.571,468.18 286.628,472.478 279.402,474.928 263.467,480.636 259.233,482.186 242.087,488.795 239.064,489.985 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#4c0c6a; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,428.166 252.915,423.529 259.233,421.453 278.728,415.371 279.402,415.164 299.571,409.316 307.257,407.213 319.74,403.857 338.743,399.054 339.909,398.765 \n",
       "  360.078,394.074 374.739,390.896 380.247,389.722 400.416,385.738 416.892,382.738 420.585,382.077 440.755,378.774 460.924,375.778 469.912,374.58 481.093,373.114 \n",
       "  501.262,370.772 521.431,368.733 541.6,366.995 549.663,366.421 561.769,365.574 581.938,364.459 602.107,363.642 622.276,363.122 642.445,362.899 662.614,362.973 \n",
       "  682.783,363.345 702.952,364.014 723.121,364.98 743.29,366.243 745.599,366.421 763.459,367.826 783.628,369.715 803.797,371.905 823.966,374.398 825.276,374.58 \n",
       "  844.135,377.237 864.304,380.387 878.023,382.738 884.473,383.862 904.642,387.691 920.253,390.896 924.811,391.848 944.98,396.38 956.104,399.054 965.149,401.268 \n",
       "  985.318,406.527 987.794,407.213 1005.49,412.199 1016.13,415.371 1025.66,418.265 1042.09,423.529 1045.83,424.747 1065.99,431.67 1066.04,431.687 1086.16,439.074 \n",
       "  1088.17,439.845 1106.33,446.962 1108.88,448.004 1126.5,455.365 1128.33,456.162 1146.67,464.313 1146.69,464.32 1163.96,472.478 1166.84,473.869 1180.34,480.636 \n",
       "  1187.01,484.055 1195.92,488.795 1207.18,494.914 1210.8,496.953 1224.95,505.111 1227.35,506.522 1238.4,513.269 1247.52,518.966 1251.32,521.427 1263.65,529.586 \n",
       "  1267.68,532.323 1275.42,537.744 1286.77,545.902 1287.85,546.7 1297.53,554.06 1307.97,562.219 1308.02,562.265 1317.82,570.377 1327.4,578.535 1328.19,579.227 \n",
       "  1336.46,586.693 1345.24,594.851 1348.36,597.845 1353.6,603.01 1361.61,611.168 1368.53,618.448 1369.34,619.326 1376.63,627.484 1383.68,635.642 1388.7,641.671 \n",
       "  1390.42,643.801 1396.79,651.959 1402.92,660.117 1408.8,668.275 1408.87,668.367 1414.3,676.433 1419.56,684.592 1424.58,692.75 1429.04,700.351 1429.36,700.908 \n",
       "  1433.79,709.066 1437.99,717.225 1441.96,725.383 1445.7,733.541 1449.21,741.675 1449.22,741.699 1452.42,749.857 1455.39,758.016 1458.14,766.174 1460.67,774.332 \n",
       "  1462.97,782.49 1465.05,790.648 1466.9,798.807 1468.53,806.965 1469.38,811.89 1469.92,815.123 1471.07,823.281 1472,831.439 1472.71,839.598 1473.2,847.756 \n",
       "  1473.48,855.914 1473.53,864.072 1473.37,872.231 1472.99,880.389 1472.38,888.547 1471.56,896.705 1470.52,904.863 1469.38,912.279 1469.26,913.022 1467.74,921.18 \n",
       "  1466,929.338 1464.04,937.496 1461.85,945.654 1459.43,953.813 1456.8,961.971 1453.93,970.129 1450.84,978.287 1449.21,982.322 1447.49,986.445 1443.86,994.604 \n",
       "  1440.01,1002.76 1435.92,1010.92 1431.6,1019.08 1429.04,1023.68 1427,1027.24 1422.1,1035.39 1416.96,1043.55 1411.58,1051.71 1408.87,1055.66 1405.89,1059.87 \n",
       "  1399.89,1068.03 1393.64,1076.19 1388.7,1082.4 1387.11,1084.34 1380.19,1092.5 1373.02,1100.66 1368.53,1105.6 1365.52,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#5e126e; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1454.96,1108.82 1461.64,1100.66 1468.1,1092.5 1469.38,1090.83 1474.21,1084.34 1480.07,1076.19 1485.71,1068.03 1489.54,1062.26 1491.1,1059.87 1496.18,1051.71 \n",
       "  1501.05,1043.55 1505.7,1035.39 1509.71,1028.02 1510.13,1027.24 1514.26,1019.08 1518.17,1010.92 1521.88,1002.76 1525.38,994.604 1528.67,986.445 1529.88,983.249 \n",
       "  1531.71,978.287 1534.52,970.129 1537.13,961.971 1539.53,953.813 1541.72,945.654 1543.71,937.496 1545.5,929.338 1547.08,921.18 1548.46,913.022 1549.64,904.863 \n",
       "  1550.05,901.387 1550.6,896.705 1551.34,888.547 1551.89,880.389 1552.24,872.231 1552.39,864.072 1552.34,855.914 1552.09,847.756 1551.64,839.598 1551,831.439 \n",
       "  1550.15,823.281 1550.05,822.542 1549.08,815.123 1547.8,806.965 1546.32,798.807 1544.63,790.648 1542.74,782.49 1540.65,774.332 1538.35,766.174 1535.85,758.016 \n",
       "  1533.14,749.857 1530.23,741.699 1529.88,740.779 1527.06,733.541 1523.66,725.383 1520.05,717.225 1516.24,709.066 1512.22,700.908 1509.71,696.08 1507.94,692.75 \n",
       "  1503.4,684.592 1498.64,676.433 1493.67,668.275 1489.54,661.794 1488.45,660.117 1482.92,651.959 1477.17,643.801 1471.2,635.642 1469.38,633.241 1464.9,627.484 \n",
       "  1458.33,619.326 1451.54,611.168 1449.21,608.457 1444.4,603.01 1436.98,594.851 1429.32,586.693 1429.04,586.402 1421.23,578.535 1412.9,570.377 1408.87,566.533 \n",
       "  1404.22,562.219 1395.18,554.06 1388.7,548.365 1385.82,545.902 1376.03,537.744 1368.53,531.644 1365.93,529.586 1355.35,521.427 1348.36,516.165 1344.4,513.269 \n",
       "  1332.99,505.111 1328.19,501.758 1321.11,496.953 1308.81,488.795 1308.02,488.286 1295.87,480.636 1287.85,475.698 1282.46,472.478 1268.5,464.32 1267.68,463.851 \n",
       "  1253.82,456.162 1247.52,452.738 1238.5,448.004 1227.35,442.255 1222.5,439.845 1207.18,432.366 1205.73,431.687 1188.07,423.529 1187.01,423.046 1169.43,415.371 \n",
       "  1166.84,414.258 1149.76,407.213 1146.67,405.961 1128.92,399.054 1126.5,398.129 1106.77,390.896 1106.33,390.738 1086.16,383.784 1082.99,382.738 1065.99,377.237 \n",
       "  1057.38,374.58 1045.83,371.074 1029.69,366.421 1025.66,365.277 1005.49,359.852 999.237,358.263 985.318,354.782 965.47,350.105 965.149,350.031 944.98,345.641 \n",
       "  926.813,341.947 924.811,341.546 904.642,337.792 884.473,334.321 881.107,333.789 864.304,331.174 844.135,328.314 823.966,325.733 823.067,325.63 803.797,323.464 \n",
       "  783.628,321.472 763.459,319.755 743.29,318.312 728.787,317.472 723.121,317.149 702.952,316.269 682.783,315.66 662.614,315.322 642.445,315.254 622.276,315.457 \n",
       "  602.107,315.931 581.938,316.675 566.104,317.472 561.769,317.694 541.6,318.999 521.431,320.579 501.262,322.434 481.093,324.564 472.144,325.63 460.924,326.989 \n",
       "  440.755,329.709 420.585,332.708 413.938,333.789 400.416,336.021 380.247,339.633 368.271,341.947 360.078,343.555 339.909,347.8 329.654,350.105 319.74,352.369 \n",
       "  299.571,357.268 295.703,358.263 279.402,362.527 265.309,366.421 259.233,368.128 239.064,374.096 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#71196d; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,325.456 259.233,320.029 269.243,317.472 279.402,314.915 299.571,310.11 303.113,309.314 319.74,305.631 339.909,301.43 341.313,301.156 360.078,297.548 \n",
       "  380.247,293.933 385.877,292.998 400.416,290.616 420.585,287.571 440.36,284.839 440.755,284.786 460.924,282.294 481.093,280.059 501.262,278.078 517.602,276.681 \n",
       "  521.431,276.358 541.6,274.909 561.769,273.713 581.938,272.768 602.107,272.075 622.276,271.634 642.445,271.445 662.614,271.508 682.783,271.823 702.952,272.39 \n",
       "  723.121,273.209 743.29,274.279 763.459,275.602 777.275,276.681 783.628,277.184 803.797,279.037 823.966,281.145 844.135,283.508 854.388,284.839 864.304,286.145 \n",
       "  884.473,289.061 904.642,292.235 909.119,292.998 924.811,295.708 944.98,299.454 953.542,301.156 965.149,303.497 985.318,307.831 991.817,309.314 1005.49,312.479 \n",
       "  1025.66,317.419 1025.86,317.472 1045.83,322.708 1056.42,325.63 1065.99,328.314 1084.62,333.789 1086.16,334.25 1106.33,340.554 1110.6,341.947 1126.5,347.224 \n",
       "  1134.83,350.105 1146.67,354.27 1157.58,358.263 1166.84,361.71 1179.02,366.421 1187.01,369.563 1199.3,374.58 1207.18,377.851 1218.53,382.738 1227.35,386.597 \n",
       "  1236.83,390.896 1247.52,395.823 1254.29,399.054 1267.68,405.556 1270.99,407.213 1286.97,415.371 1287.85,415.831 1302.2,423.529 1308.02,426.712 1316.85,431.687 \n",
       "  1328.19,438.202 1330.97,439.845 1344.49,448.004 1348.36,450.383 1357.49,456.162 1368.53,463.296 1370.07,464.32 1382.1,472.478 1388.7,477.047 1393.74,480.636 \n",
       "  1404.97,488.795 1408.87,491.694 1415.76,496.953 1426.22,505.111 1429.04,507.363 1436.25,513.269 1445.98,521.427 1449.21,524.202 1455.32,529.586 1464.36,537.744 \n",
       "  1469.38,542.388 1473.08,545.902 1481.46,554.06 1489.54,562.141 1489.62,562.219 1497.37,570.377 1504.92,578.535 1509.71,583.877 1512.18,586.693 1519.13,594.851 \n",
       "  1525.87,603.01 1529.88,608.019 1532.35,611.168 1538.52,619.326 1544.5,627.484 1550.05,635.335 1550.26,635.642 1555.71,643.801 1560.95,651.959 1565.99,660.117 \n",
       "  1570.22,667.236 1570.82,668.275 1575.37,676.433 1579.72,684.592 1583.87,692.75 1587.83,700.908 1590.39,706.468 1591.56,709.066 1595.05,717.225 1598.35,725.383 \n",
       "  1601.46,733.541 1604.38,741.699 1607.1,749.857 1609.64,758.016 1610.56,761.218 1611.95,766.174 1614.06,774.332 1615.98,782.49 1617.71,790.648 1619.26,798.807 \n",
       "  1620.62,806.965 1621.79,815.123 1622.77,823.281 1623.57,831.439 1624.18,839.598 1624.6,847.756 1624.83,855.914 1624.88,864.072 1624.74,872.231 1624.41,880.389 \n",
       "  1623.89,888.547 1623.19,896.705 1622.3,904.863 1621.22,913.022 1619.96,921.18 1618.51,929.338 1616.87,937.496 1615.04,945.654 1613.03,953.813 1610.83,961.971 \n",
       "  1610.56,962.893 1608.4,970.129 1605.77,978.287 1602.94,986.445 1599.93,994.604 1596.73,1002.76 1593.33,1010.92 1590.39,1017.61 1589.73,1019.08 1585.87,1027.24 \n",
       "  1581.82,1035.39 1577.57,1043.55 1573.12,1051.71 1570.22,1056.81 1568.44,1059.87 1563.5,1068.03 1558.35,1076.19 1553.01,1084.34 1550.05,1088.7 1547.41,1092.5 \n",
       "  1541.54,1100.66 1535.46,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#83206b; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1609.27,1108.82 1610.56,1106.98 1614.87,1100.66 1620.26,1092.5 1625.46,1084.34 1630.47,1076.19 1630.73,1075.75 1635.2,1068.03 1639.74,1059.87 1644.1,1051.71 \n",
       "  1648.27,1043.55 1650.9,1038.19 1652.24,1035.39 1655.97,1027.24 1659.52,1019.08 1662.89,1010.92 1666.09,1002.76 1669.1,994.604 1671.07,988.944 1671.92,986.445 \n",
       "  1674.52,978.287 1676.94,970.129 1679.19,961.971 1681.26,953.813 1683.16,945.654 1684.88,937.496 1686.42,929.338 1687.79,921.18 1688.98,913.022 1689.99,904.863 \n",
       "  1690.83,896.705 1691.23,891.72 1691.49,888.547 1691.96,880.389 1692.27,872.231 1692.4,864.072 1692.35,855.914 1692.14,847.756 1691.75,839.598 1691.23,832.171 \n",
       "  1691.18,831.439 1690.43,823.281 1689.51,815.123 1688.41,806.965 1687.13,798.807 1685.67,790.648 1684.04,782.49 1682.23,774.332 1680.25,766.174 1678.09,758.016 \n",
       "  1675.75,749.857 1673.24,741.699 1671.07,735.106 1670.54,733.541 1667.62,725.383 1664.51,717.225 1661.23,709.066 1657.77,700.908 1654.13,692.75 1650.9,685.856 \n",
       "  1650.29,684.592 1646.21,676.433 1641.94,668.275 1637.49,660.117 1632.86,651.959 1630.73,648.351 1627.99,643.801 1622.88,635.642 1617.59,627.484 1612.11,619.326 \n",
       "  1610.56,617.093 1606.36,611.168 1600.38,603.01 1594.21,594.851 1590.39,589.951 1587.79,586.693 1581.1,578.535 1574.21,570.377 1570.22,565.779 1567.06,562.219 \n",
       "  1559.62,554.06 1551.98,545.902 1550.05,543.895 1544.01,537.744 1535.79,529.586 1529.88,523.87 1527.3,521.427 1518.47,513.269 1509.71,505.363 1509.43,505.111 \n",
       "  1499.96,496.953 1490.28,488.795 1489.54,488.187 1480.17,480.636 1469.81,472.478 1469.38,472.141 1458.99,464.32 1449.21,457.104 1447.9,456.162 1436.33,448.004 \n",
       "  1429.04,442.965 1424.41,439.845 1412.06,431.687 1408.87,429.618 1399.22,423.529 1388.7,417.005 1385.99,415.371 1372.21,407.213 1368.53,405.07 1357.89,399.054 \n",
       "  1348.36,393.756 1343.07,390.896 1328.19,383.002 1327.68,382.738 1311.57,374.58 1308.02,372.811 1294.8,366.421 1287.85,363.121 1277.31,358.263 1267.68,353.904 \n",
       "  1259.02,350.105 1247.52,345.137 1239.87,341.947 1227.35,336.799 1219.76,333.789 1207.18,328.871 1198.58,325.63 1187.01,321.334 1176.21,317.472 1166.84,314.171 \n",
       "  1152.51,309.314 1146.67,307.364 1127.31,301.156 1126.5,300.9 1106.33,294.787 1100.16,292.998 1086.16,288.996 1070.94,284.839 1065.99,283.508 1045.83,278.334 \n",
       "  1039.05,276.681 1025.66,273.46 1005.49,268.862 1003.92,268.523 985.318,264.57 965.149,260.531 964.263,260.365 944.98,256.79 924.811,253.296 918.048,252.207 \n",
       "  904.642,250.076 884.473,247.112 864.304,244.391 861.52,244.048 844.135,241.939 823.966,239.73 803.797,237.761 783.628,236.03 781.743,235.89 763.459,234.555 \n",
       "  743.29,233.318 723.121,232.316 702.952,231.55 682.783,231.02 662.614,230.725 642.445,230.666 622.276,230.843 602.107,231.256 581.938,231.904 561.769,232.787 \n",
       "  541.6,233.907 521.431,235.262 513.465,235.89 501.262,236.865 481.093,238.716 460.924,240.805 440.755,243.133 433.559,244.048 420.585,245.721 400.416,248.564 \n",
       "  380.247,251.648 376.864,252.207 360.078,255.012 339.909,258.629 330.843,260.365 319.74,262.52 299.571,266.682 291.157,268.523 279.402,271.13 259.233,275.854 \n",
       "  255.881,276.681 239.064,280.889 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#972765; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,239.491 254.468,235.89 259.233,234.79 279.402,230.372 292.13,227.732 299.571,226.208 319.74,222.311 334.809,219.574 339.909,218.659 360.078,215.27 \n",
       "  380.247,212.112 385.042,211.415 400.416,209.211 420.585,206.545 440.755,204.106 448.497,203.257 460.924,201.911 481.093,199.951 501.262,198.214 521.431,196.702 \n",
       "  541.6,195.413 547.557,195.099 561.769,194.358 581.938,193.528 602.107,192.92 622.276,192.532 642.445,192.366 662.614,192.421 682.783,192.698 702.952,193.196 \n",
       "  723.121,193.916 743.29,194.856 747.504,195.099 763.459,196.03 783.628,197.43 803.797,199.054 823.966,200.903 844.135,202.976 846.609,203.257 864.304,205.297 \n",
       "  884.473,207.849 904.642,210.628 909.923,211.415 924.811,213.662 944.98,216.936 960.167,219.574 965.149,220.45 985.318,224.231 1002.91,227.732 1005.49,228.251 \n",
       "  1025.66,232.552 1040.5,235.89 1045.83,237.104 1065.99,241.939 1074.38,244.048 1086.16,247.052 1105.48,252.207 1106.33,252.438 1126.5,258.138 1134.05,260.365 \n",
       "  1146.67,264.135 1160.77,268.523 1166.84,270.436 1185.88,276.681 1187.01,277.056 1207.18,284.019 1209.47,284.839 1227.35,291.328 1231.79,292.998 1247.52,298.993 \n",
       "  1253,301.156 1267.68,307.031 1273.21,309.314 1287.85,315.456 1292.51,317.472 1308.02,324.288 1310.99,325.63 1328.19,333.545 1328.71,333.789 1345.66,341.947 \n",
       "  1348.36,343.266 1361.96,350.105 1368.53,353.465 1377.66,358.263 1388.7,364.161 1392.82,366.421 1407.44,374.58 1408.87,375.393 1421.47,382.738 1429.04,387.221 \n",
       "  1435.08,390.896 1448.26,399.054 1449.21,399.649 1460.92,407.213 1469.38,412.774 1473.23,415.371 1485.11,423.529 1489.54,426.626 1496.61,431.687 1507.79,439.845 \n",
       "  1509.71,441.278 1518.54,448.004 1529.04,456.162 1529.88,456.826 1539.12,464.32 1548.98,472.478 1550.05,473.386 1558.44,480.636 1567.68,488.795 1570.22,491.09 \n",
       "  1576.57,496.953 1585.22,505.111 1590.39,510.098 1593.6,513.269 1601.69,521.427 1609.58,529.586 1610.56,530.625 1617.14,537.744 1624.49,545.902 1630.73,553.005 \n",
       "  1631.64,554.06 1638.47,562.219 1645.12,570.377 1650.9,577.656 1651.58,578.535 1657.74,586.693 1663.72,594.851 1669.52,603.01 1671.07,605.251 1675.06,611.168 \n",
       "  1680.4,619.326 1685.56,627.484 1690.54,635.642 1691.23,636.826 1695.26,643.801 1699.81,651.959 1704.17,660.117 1708.37,668.275 1711.4,674.433 1712.37,676.433 \n",
       "  1716.15,684.592 1719.75,692.75 1723.19,700.908 1726.46,709.066 1729.56,717.225 1731.57,722.842 1732.47,725.383 1735.18,733.541 1737.72,741.699 1740.09,749.857 \n",
       "  1742.3,758.016 1744.34,766.174 1746.21,774.332 1747.92,782.49 1749.46,790.648 1750.84,798.807 1751.74,804.925 1752.04,806.965 1753.06,815.123 1753.92,823.281 \n",
       "  1754.61,831.439 1755.15,839.598 1755.51,847.756 1755.72,855.914 1755.76,864.072 1755.64,872.231 1755.35,880.389 1754.9,888.547 1754.29,896.705 1753.51,904.863 \n",
       "  1752.57,913.022 1751.74,919.14 1751.46,921.18 1750.17,929.338 1748.71,937.496 1747.09,945.654 1745.3,953.813 1743.34,961.971 1741.21,970.129 1738.92,978.287 \n",
       "  1736.47,986.445 1733.84,994.604 1731.57,1001.24 1731.04,1002.76 1728.03,1010.92 1724.85,1019.08 1721.49,1027.24 1717.97,1035.39 1714.28,1043.55 1711.4,1049.63 \n",
       "  1710.4,1051.71 1706.29,1059.87 1702.01,1068.03 1697.56,1076.19 1692.93,1084.34 1691.23,1087.22 1688.07,1092.5 1683,1100.66 1677.75,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#a72d5f; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1741.98,1108.82 1746.94,1100.66 1751.73,1092.5 1751.74,1092.47 1756.27,1084.34 1760.64,1076.19 1764.85,1068.03 1768.9,1059.87 1771.91,1053.55 1772.77,1051.71 \n",
       "  1776.43,1043.55 1779.92,1035.39 1783.26,1027.24 1786.43,1019.08 1789.44,1010.92 1792.08,1003.38 1792.29,1002.76 1794.94,994.604 1797.42,986.445 1799.75,978.287 \n",
       "  1801.92,970.129 1803.93,961.971 1805.79,953.813 1807.49,945.654 1809.03,937.496 1810.41,929.338 1811.63,921.18 1812.25,916.445 1812.69,913.022 1813.58,904.863 \n",
       "  1814.32,896.705 1814.9,888.547 1815.33,880.389 1815.6,872.231 1815.71,864.072 1815.68,855.914 1815.48,847.756 1815.13,839.598 1814.63,831.439 1813.97,823.281 \n",
       "  1813.15,815.123 1812.25,807.51 1812.18,806.965 1811.04,798.807 1809.74,790.648 1808.28,782.49 1806.66,774.332 1804.88,766.174 1802.95,758.016 1800.86,749.857 \n",
       "  1798.61,741.699 1796.2,733.541 1793.63,725.383 1792.08,720.73 1790.89,717.225 1787.96,709.066 1784.86,700.908 1781.61,692.75 1778.19,684.592 1774.62,676.433 \n",
       "  1771.91,670.517 1770.87,668.275 1766.9,660.117 1762.77,651.959 1758.47,643.801 1754.02,635.642 1751.74,631.628 1749.35,627.484 1744.48,619.326 1739.44,611.168 \n",
       "  1734.23,603.01 1731.57,598.972 1728.81,594.851 1723.16,586.693 1717.35,578.535 1711.4,570.425 1711.37,570.377 1705.1,562.219 1698.65,554.06 1692.04,545.902 \n",
       "  1691.23,544.938 1685.13,537.744 1678.03,529.586 1671.07,521.773 1670.75,521.427 1663.15,513.269 1655.37,505.111 1650.9,500.523 1647.34,496.953 1639.04,488.795 \n",
       "  1630.73,480.806 1630.55,480.636 1621.7,472.478 1612.66,464.32 1610.56,462.463 1603.28,456.162 1593.67,448.004 1590.39,445.274 1583.73,439.845 1573.51,431.687 \n",
       "  1570.22,429.105 1562.95,423.529 1552.12,415.371 1550.05,413.845 1540.88,407.213 1529.88,399.406 1529.38,399.054 1517.41,390.896 1509.71,385.736 1505.13,382.738 \n",
       "  1492.46,374.58 1489.54,372.735 1479.33,366.421 1469.38,360.371 1465.82,358.263 1451.84,350.105 1449.21,348.591 1437.35,341.947 1429.04,337.366 1422.38,333.789 \n",
       "  1408.87,326.639 1406.91,325.63 1390.84,317.472 1388.7,316.404 1374.13,309.314 1368.53,306.63 1356.79,301.156 1348.36,297.284 1338.76,292.998 1328.19,288.348 \n",
       "  1319.98,284.839 1308.02,279.803 1300.38,276.681 1287.85,271.633 1279.89,268.523 1267.68,263.824 1258.4,260.365 1247.52,256.36 1235.83,252.207 1227.35,249.229 \n",
       "  1212.06,244.048 1207.18,242.416 1187.01,235.91 1186.95,235.89 1166.84,229.723 1160.09,227.732 1146.67,223.823 1131.48,219.574 1126.5,218.199 1106.33,212.858 \n",
       "  1100.64,211.415 1086.16,207.793 1067.19,203.257 1065.99,202.975 1045.83,198.438 1030.21,195.099 1025.66,194.137 1005.49,190.097 988.814,186.941 985.318,186.287 \n",
       "  965.149,182.734 944.98,179.399 940.993,178.783 924.811,176.312 904.642,173.449 884.473,170.802 883,170.624 864.304,168.397 844.135,166.208 823.966,164.233 \n",
       "  803.797,162.471 803.73,162.466 783.628,160.941 763.459,159.622 743.29,158.514 723.121,157.617 702.952,156.931 682.783,156.456 662.614,156.192 642.445,156.139 \n",
       "  622.276,156.297 602.107,156.667 581.938,157.247 561.769,158.039 541.6,159.041 521.431,160.255 501.262,161.68 491.565,162.466 481.093,163.326 460.924,165.194 \n",
       "  440.755,167.276 420.585,169.572 412.126,170.624 400.416,172.098 380.247,174.854 360.078,177.825 354.017,178.783 339.909,181.039 319.74,184.483 306.207,186.941 \n",
       "  299.571,188.161 279.402,192.089 264.775,195.099 259.233,196.253 239.064,200.679 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#ba3655; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,164.019 246.492,162.466 259.233,159.833 279.402,155.875 287.838,154.308 299.571,152.153 319.74,148.658 335.137,146.15 339.909,145.381 360.078,142.338 \n",
       "  380.247,139.502 391.827,137.992 400.416,136.884 420.585,134.487 440.755,132.294 460.924,130.306 466.26,129.833 481.093,128.535 501.262,126.972 521.431,125.61 \n",
       "  541.6,124.451 561.769,123.492 581.938,122.736 602.107,122.181 622.276,121.828 642.445,121.677 662.614,121.727 682.783,121.979 702.952,122.433 723.121,123.089 \n",
       "  743.29,123.946 763.459,125.005 783.628,126.266 803.797,127.728 823.966,129.393 828.73,129.833 844.135,131.275 864.304,133.365 884.473,135.66 903.292,137.992 \n",
       "  904.642,138.161 924.811,140.894 944.98,143.834 959.829,146.15 965.149,146.989 985.318,150.38 1005.49,153.979 1007.23,154.308 1025.66,157.827 1045.83,161.891 \n",
       "  1048.54,162.466 1065.99,166.208 1085.62,170.624 1086.16,170.748 1106.33,175.556 1119.29,178.783 1126.5,180.601 1146.67,185.904 1150.46,186.941 1166.84,191.48 \n",
       "  1179.4,195.099 1187.01,197.318 1206.63,203.257 1207.18,203.425 1227.35,209.834 1232.15,211.415 1247.52,216.533 1256.34,219.574 1267.68,223.532 1279.33,227.732 \n",
       "  1287.85,230.843 1301.25,235.89 1308.02,238.476 1322.18,244.048 1328.19,246.446 1342.21,252.207 1348.36,254.767 1361.43,260.365 1368.53,263.451 1379.88,268.523 \n",
       "  1388.7,272.515 1397.65,276.681 1408.87,281.974 1414.78,284.839 1429.04,291.846 1431.32,292.998 1447.27,301.156 1449.21,302.162 1462.64,309.314 1469.38,312.952 \n",
       "  1477.54,317.472 1489.54,324.219 1492,325.63 1505.95,333.789 1509.71,336.02 1519.47,341.947 1529.88,348.375 1532.62,350.105 1545.34,358.263 1550.05,361.337 \n",
       "  1557.68,366.421 1569.71,374.58 1570.22,374.931 1581.3,382.738 1590.39,389.252 1592.63,390.896 1603.59,399.054 1610.56,404.341 1614.27,407.213 1624.62,415.371 \n",
       "  1630.73,420.277 1634.7,423.529 1644.47,431.687 1650.9,437.156 1653.99,439.845 1663.21,448.004 1671.07,455.092 1672.23,456.162 1680.91,464.32 1689.42,472.478 \n",
       "  1691.23,474.252 1697.63,480.636 1705.63,488.795 1711.4,494.811 1713.42,496.953 1720.93,505.111 1728.27,513.269 1731.57,517.025 1735.37,521.427 1742.24,529.586 \n",
       "  1748.95,537.744 1751.74,541.227 1755.42,545.902 1761.68,554.06 1767.77,562.219 1771.91,567.913 1773.67,570.377 1779.33,578.535 1784.84,586.693 1790.18,594.851 \n",
       "  1792.08,597.842 1795.3,603.01 1800.24,611.168 1805.01,619.326 1809.63,627.484 1812.25,632.272 1814.06,635.642 1818.29,643.801 1822.36,651.959 1826.28,660.117 \n",
       "  1830.04,668.275 1832.42,673.649 1833.63,676.433 1837.02,684.592 1840.26,692.75 1843.35,700.908 1846.29,709.066 1849.07,717.225 1851.7,725.383 1852.59,728.285 \n",
       "  1854.16,733.541 1856.44,741.699 1858.58,749.857 1860.57,758.016 1862.41,766.174 1864.1,774.332 1865.63,782.49 1867.02,790.648 1868.26,798.807 1869.35,806.965 \n",
       "  1870.28,815.123 1871.07,823.281 1871.71,831.439 1872.2,839.598 1872.53,847.756 1872.72,855.914 1872.76,863.444 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#ba3655; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,864.282 1872.65,872.231 1872.38,880.389 1871.97,888.547 1871.41,896.705 1870.7,904.863 1869.83,913.022 1868.82,921.18 1867.66,929.338 1866.35,937.496 \n",
       "  1864.88,945.654 1863.27,953.813 1861.51,961.971 1859.6,970.129 1857.53,978.287 1855.32,986.445 1852.96,994.604 1852.59,995.806 1850.41,1002.76 1847.7,1010.92 \n",
       "  1844.84,1019.08 1841.83,1027.24 1838.66,1035.39 1835.34,1043.55 1832.42,1050.43 1831.86,1051.71 1828.18,1059.87 1824.34,1068.03 1820.34,1076.19 1816.19,1084.34 \n",
       "  1812.25,1091.82 1811.88,1092.5 1807.34,1100.66 1802.65,1108.82 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#cb4148; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1860.28,1108.82 1864.75,1100.66 1869.06,1092.5 1872.76,1085.26 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#cb4148; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,638.828 1871.16,635.642 1866.92,627.484 1862.53,619.326 1858,611.168 1853.31,603.01 1852.59,601.793 1848.4,594.851 1843.33,586.693 1838.1,578.535 \n",
       "  1832.73,570.377 1832.42,569.921 1827.11,562.219 1821.33,554.06 1815.39,545.902 1812.25,541.691 1809.25,537.744 1802.9,529.586 1796.38,521.427 1792.08,516.163 \n",
       "  1789.67,513.269 1782.72,505.111 1775.61,496.953 1771.91,492.799 1768.28,488.795 1760.71,480.636 1752.98,472.478 1751.74,471.197 1744.97,464.32 1736.76,456.162 \n",
       "  1731.57,451.106 1728.33,448.004 1719.63,439.845 1711.4,432.278 1710.75,431.687 1701.54,423.529 1692.15,415.371 1691.23,414.586 1682.43,407.213 1672.51,399.054 \n",
       "  1671.07,397.889 1662.24,390.896 1651.76,382.738 1650.9,382.075 1640.91,374.58 1630.73,367.069 1629.83,366.421 1618.36,358.263 1610.56,352.806 1606.62,350.105 \n",
       "  1594.52,341.947 1590.39,339.207 1582.05,333.789 1570.22,326.22 1569.28,325.63 1556.05,317.472 1550.05,313.832 1542.45,309.314 1529.88,301.962 1528.47,301.156 \n",
       "  1514,292.998 1509.71,290.615 1499.07,284.839 1489.54,279.738 1483.69,276.681 1469.38,269.302 1467.83,268.523 1451.38,260.365 1449.21,259.302 1434.33,252.207 \n",
       "  1429.04,249.712 1416.69,244.048 1408.87,240.506 1398.4,235.89 1388.7,231.667 1379.41,227.732 1368.53,223.183 1359.65,219.574 1348.36,215.04 1339.07,211.415 \n",
       "  1328.19,207.225 1317.58,203.257 1308.02,199.726 1295.11,195.099 1287.85,192.532 1271.55,186.941 1267.68,185.631 1247.52,179.016 1246.78,178.783 1227.35,172.692 \n",
       "  1220.51,170.624 1207.18,166.635 1192.73,162.466 1187.01,160.835 1166.84,155.294 1163.11,154.308 1146.67,150.015 1131.27,146.15 1126.5,144.968 1106.33,140.172 \n",
       "  1096.75,137.992 1086.16,135.609 1065.99,131.274 1058.96,129.833 1045.83,127.174 1025.66,123.29 1016.81,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#d84c3d; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1169.91,121.675 1187.01,126.165 1200.47,129.833 1207.18,131.682 1227.35,137.445 1229.2,137.992 1247.52,143.472 1256.17,146.15 1267.68,149.754 1281.77,154.308 \n",
       "  1287.85,156.297 1306.13,162.466 1308.02,163.112 1328.19,170.212 1329.33,170.624 1348.36,177.608 1351.47,178.783 1368.53,185.303 1372.7,186.941 1388.7,193.306 \n",
       "  1393.08,195.099 1408.87,201.63 1412.7,203.257 1429.04,210.287 1431.59,211.415 1449.21,219.29 1449.83,219.574 1467.4,227.732 1469.38,228.662 1484.37,235.89 \n",
       "  1489.54,238.416 1500.81,244.048 1509.71,248.563 1516.74,252.207 1529.88,259.118 1532.2,260.365 1547.16,268.523 1550.05,270.121 1561.66,276.681 1570.22,281.591 \n",
       "  1575.77,284.839 1589.49,292.998 1590.39,293.538 1602.76,301.156 1610.56,306.03 1615.71,309.314 1628.3,317.472 1630.73,319.066 1640.51,325.63 1650.9,332.707 \n",
       "  1652.45,333.789 1664.01,341.947 1671.07,347.007 1675.3,350.105 1686.28,358.263 1691.23,362.005 1696.97,366.421 1707.39,374.58 1711.4,377.773 1717.52,382.738 \n",
       "  1727.41,390.896 1731.57,394.39 1737.03,399.054 1746.4,407.213 1751.74,411.95 1755.53,415.371 1764.41,423.529 1771.91,430.557 1773.1,431.687 1781.49,439.845 \n",
       "  1789.73,448.004 1792.08,450.381 1797.7,456.162 1805.48,464.32 1812.25,471.574 1813.08,472.478 1820.41,480.636 1827.59,488.795 1832.42,494.408 1834.57,496.953 \n",
       "  1841.32,505.111 1847.92,513.269 1852.59,519.179 1854.33,521.427 1860.52,529.586 1866.56,537.744 1872.45,545.902 1872.76,546.34 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#e55c2f; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,475.407 1870.21,472.478 1862.97,464.32 1855.58,456.162 1852.59,452.917 1847.97,448.004 1840.15,439.845 1832.42,431.929 1832.18,431.687 1823.92,423.529 \n",
       "  1815.5,415.371 1812.25,412.279 1806.83,407.213 1797.95,399.054 1792.08,393.754 1788.86,390.896 1779.5,382.738 1771.91,376.237 1769.94,374.58 1760.09,366.421 \n",
       "  1751.74,359.627 1750.04,358.263 1739.66,350.105 1731.57,343.841 1729.08,341.947 1718.18,333.789 1711.4,328.8 1707.02,325.63 1695.56,317.472 1691.23,314.44 \n",
       "  1683.78,309.314 1671.75,301.156 1671.07,300.701 1659.3,292.998 1650.9,287.569 1646.58,284.839 1633.51,276.681 1630.73,274.971 1620.03,268.523 1610.56,262.891 \n",
       "  1606.22,260.365 1592.02,252.207 1590.39,251.284 1577.35,244.048 1570.22,240.147 1562.27,235.89 1550.05,229.428 1546.77,227.732 1530.79,219.574 1529.88,219.117 \n",
       "  1514.25,211.415 1509.71,209.21 1497.19,203.257 1489.54,199.67 1479.57,195.099 1469.38,190.484 1461.35,186.941 1449.21,181.639 1442.49,178.783 1429.04,173.124 \n",
       "  1422.94,170.624 1408.87,164.927 1402.63,162.466 1388.7,157.035 1381.51,154.308 1368.53,149.44 1359.51,146.15 1348.36,142.131 1336.54,137.992 1328.19,135.099 \n",
       "  1312.53,129.833 1308.02,128.333 1287.85,121.828 1287.37,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#ef6d21; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1386.33,121.675 1388.7,122.534 1408.29,129.833 1408.87,130.05 1429.04,137.852 1429.39,137.992 1449.21,145.948 1449.7,146.15 1469.29,154.308 1469.38,154.344 \n",
       "  1488.19,162.466 1489.54,163.058 1506.46,170.624 1509.71,172.098 1524.14,178.783 1529.88,181.475 1541.27,186.941 1550.05,191.203 1557.9,195.099 1570.22,201.294 \n",
       "  1574.04,203.257 1589.72,211.415 1590.39,211.766 1604.9,219.574 1610.56,222.659 1619.67,227.732 1630.73,233.964 1634.07,235.89 1648.07,244.048 1650.9,245.72 \n",
       "  1661.66,252.207 1671.07,257.953 1674.94,260.365 1687.85,268.523 1691.23,270.687 1700.42,276.681 1711.4,283.954 1712.72,284.839 1724.64,292.998 1731.57,297.809 \n",
       "  1736.31,301.156 1747.68,309.314 1751.74,312.274 1758.75,317.472 1769.59,325.63 1771.91,327.405 1780.11,333.789 1790.44,341.947 1792.08,343.265 1800.45,350.105 \n",
       "  1810.28,358.263 1812.25,359.924 1819.82,366.421 1829.17,374.58 1832.42,377.465 1838.26,382.738 1847.14,390.896 1852.59,395.98 1855.82,399.054 1864.26,407.213 \n",
       "  1872.55,415.371 1872.76,415.577 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#f68211; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,362.896 1867.54,358.263 1858.2,350.105 1852.59,345.279 1848.65,341.947 1838.84,333.789 1832.42,328.521 1828.83,325.63 1818.55,317.472 1812.25,312.545 \n",
       "  1808.04,309.314 1797.27,301.156 1792.08,297.283 1786.23,292.998 1774.95,284.839 1771.91,282.676 1763.35,276.681 1751.74,268.671 1751.52,268.523 1739.32,260.365 \n",
       "  1731.57,255.256 1726.86,252.207 1714.09,244.048 1711.4,242.355 1700.95,235.89 1691.23,229.958 1687.52,227.732 1673.72,219.574 1671.07,218.026 1659.52,211.415 \n",
       "  1650.9,206.544 1644.96,203.257 1630.73,195.468 1630.04,195.099 1614.63,186.941 1610.56,184.81 1598.79,178.783 1590.39,174.528 1582.51,170.624 1570.22,164.606 \n",
       "  1565.75,162.466 1550.05,155.03 1548.49,154.308 1530.67,146.15 1529.88,145.793 1512.25,137.992 1509.71,136.883 1493.21,129.833 1489.54,128.282 1473.53,121.675 \n",
       "  \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#fa9607; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1552.39,121.675 1570.22,129.745 1570.41,129.833 1587.85,137.992 1590.39,139.191 1604.79,146.15 1610.56,148.97 1621.25,154.308 1630.73,159.093 1637.27,162.466 \n",
       "  1650.9,169.571 1652.88,170.624 1668.03,178.783 1671.07,180.436 1682.77,186.941 1691.23,191.701 1697.16,195.099 1711.22,203.257 1711.4,203.368 1724.84,211.415 \n",
       "  1731.57,215.499 1738.17,219.574 1751.2,227.732 1751.74,228.072 1763.85,235.89 1771.91,241.161 1776.25,244.048 1788.34,252.207 1792.08,254.765 1800.13,260.365 \n",
       "  1811.69,268.523 1812.25,268.923 1822.91,276.681 1832.42,283.698 1833.94,284.839 1844.66,292.998 1852.59,299.123 1855.18,301.156 1865.41,309.314 1872.76,315.252 \n",
       "  \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#fbac11; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,271.443 1868.88,268.523 1857.89,260.365 1852.59,256.481 1846.65,252.207 1835.17,244.048 1832.42,242.116 1823.39,235.89 1812.25,228.308 1811.39,227.732 \n",
       "  1799.03,219.574 1792.08,215.039 1786.42,211.415 1773.53,203.257 1771.91,202.246 1760.27,195.099 1751.74,189.93 1746.72,186.941 1732.85,178.783 1731.57,178.039 \n",
       "  1718.58,170.624 1711.4,166.581 1703.97,162.466 1691.23,155.505 1689,154.308 1673.62,146.15 1671.07,144.813 1657.79,137.992 1650.9,134.486 1641.56,129.833 \n",
       "  1630.73,124.5 1624.87,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#f9c32a; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1692.39,121.675 1708,129.833 1711.4,131.63 1723.22,137.992 1731.57,142.543 1738.07,146.15 1751.74,153.821 1752.59,154.308 1766.7,162.466 1771.91,165.513 \n",
       "  1780.49,170.624 1792.08,177.607 1794,178.783 1807.14,186.941 1812.25,190.151 1819.99,195.099 1832.42,203.142 1832.59,203.257 1844.83,211.415 1852.59,216.647 \n",
       "  1856.85,219.574 1868.59,227.732 1872.76,230.664 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#f5d949; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1872.76,192.364 1864.55,186.941 1852.59,179.124 1852.06,178.783 1839.2,170.624 1832.42,166.367 1826.09,162.466 1812.71,154.308 1812.25,154.03 1798.94,146.15 \n",
       "  1792.08,142.13 1784.89,137.992 1771.91,130.61 1770.52,129.833 1755.76,121.675 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#f2ee74; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1815.71,121.675 1829.71,129.833 1832.42,131.426 1843.37,137.992 1852.59,143.574 1856.77,146.15 1869.85,154.308 1872.76,156.137 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#cb4148; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  278.066,121.675 259.233,125.207 239.064,129.191 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#060419; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  570.191,643.801 561.769,644.887 541.6,648.181 522.492,651.959 521.431,652.177 501.262,657.044 490.169,660.117 481.093,662.735 464.083,668.275 460.924,669.348 \n",
       "  442.206,676.433 440.755,677.008 423.364,684.592 420.585,685.86 406.782,692.75 400.416,696.084 391.927,700.908 380.247,707.891 378.424,709.066 366.428,717.225 \n",
       "  360.078,721.793 355.428,725.383 345.47,733.541 339.909,738.396 336.366,741.699 328.188,749.857 320.585,758.016 319.74,758.995 313.924,766.174 307.854,774.332 \n",
       "  302.323,782.49 299.571,786.988 297.458,790.648 293.257,798.807 289.565,806.965 286.383,815.123 283.709,823.281 281.545,831.439 279.89,839.598 279.402,843.074 \n",
       "  278.78,847.756 278.177,855.914 278.057,864.072 278.418,872.231 279.262,880.389 279.402,881.252 280.654,888.547 282.564,896.705 284.982,904.863 287.91,913.022 \n",
       "  291.347,921.18 295.294,929.338 299.571,937.17 299.76,937.496 305.021,945.654 310.822,953.813 317.162,961.971 319.74,965.028 324.315,970.129 332.205,978.287 \n",
       "  339.909,985.713 340.721,986.445 350.372,994.604 360.078,1002.32 360.677,1002.76 372.344,1010.92 380.247,1016.15 385.016,1019.08 399.016,1027.24 400.416,1028.01 \n",
       "  414.858,1035.39 420.585,1038.19 432.612,1043.55 440.755,1047.03 452.866,1051.71 460.924,1054.7 476.478,1059.87 481.093,1061.34 501.262,1067.04 505.263,1068.03 \n",
       "  521.431,1071.85 541.6,1075.92 543.206,1076.19 561.769,1079.16 581.938,1081.71 602.107,1083.58 614.965,1084.34 622.276,1084.76 642.445,1085.25 662.614,1085.09 \n",
       "  680.917,1084.34 682.783,1084.27 702.952,1082.73 723.121,1080.52 743.29,1077.63 751.441,1076.19 763.459,1073.97 783.628,1069.56 789.64,1068.03 803.797,1064.28 \n",
       "  818.47,1059.87 823.966,1058.15 842.271,1051.71 844.135,1051.03 862.513,1043.55 864.304,1042.79 880.146,1035.39 884.473,1033.28 895.809,1027.24 904.642,1022.29 \n",
       "  909.948,1019.08 922.738,1010.92 924.811,1009.52 934.142,1002.76 944.766,994.604 944.98,994.428 954.114,986.445 962.856,978.287 965.149,975.992 970.645,970.129 \n",
       "  977.737,961.971 984.272,953.813 985.318,952.385 989.965,945.654 995.074,937.496 999.658,929.338 1003.72,921.18 1005.49,917.1 1007.16,913.022 1010.01,904.863 \n",
       "  1012.36,896.705 1014.22,888.547 1015.58,880.389 1016.44,872.231 1016.82,864.072 1016.69,855.914 1016.07,847.756 1014.96,839.598 1013.35,831.439 1011.24,823.281 \n",
       "  1008.64,815.123 1005.55,806.965 1005.49,806.824 1001.75,798.807 997.432,790.648 992.585,782.49 987.214,774.332 985.318,771.708 981.074,766.174 974.26,758.016 \n",
       "  966.891,749.857 965.149,748.065 958.559,741.699 949.521,733.541 944.98,729.695 939.533,725.383 928.592,717.225 924.811,714.56 916.428,709.066 904.642,701.744 \n",
       "  903.187,700.908 888.247,692.75 884.473,690.786 871.51,684.592 864.304,681.303 852.593,676.433 844.135,673.068 830.79,668.275 823.966,665.926 805.05,660.117 \n",
       "  803.797,659.748 783.628,654.52 772.163,651.959 763.459,650.088 743.29,646.447 725.181,643.801 723.121,643.511 702.952,641.339 682.783,639.836 662.614,639.001 \n",
       "  642.445,638.834 622.276,639.335 602.107,640.504 581.938,642.342 570.191,643.801 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#ff0000; stroke-width:8; stroke-opacity:1; fill:none\" points=\"\n",
       "  953.804,1108.82 953.804,-371.897 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0702)\" style=\"stroke:#000000; stroke-width:8; stroke-opacity:1; fill:none\" points=\"\n",
       "  239.064,417.818 1872.76,417.818 \n",
       "  \"/>\n",
       "<circle clip-path=\"url(#clip0702)\" cx=\"953.804\" cy=\"862.033\" r=\"14\" fill=\"#ff0000\" fill-rule=\"evenodd\" fill-opacity=\"1\" stroke=\"#000000\" stroke-opacity=\"1\" stroke-width=\"2.56\"/>\n",
       "<path clip-path=\"url(#clip0700)\" d=\"\n",
       "M1438.61 418.315 L1815.16 418.315 L1815.16 176.395 L1438.61 176.395  Z\n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1438.61,418.315 1815.16,418.315 1815.16,176.395 1438.61,176.395 1438.61,418.315 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#ff0000; stroke-width:8; stroke-opacity:1; fill:none\" points=\"\n",
       "  1457.81,236.875 1573.01,236.875 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 1592.21, 254.375)\" x=\"1592.21\" y=\"254.375\">2.75 &lt; x</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip0700)\" style=\"stroke:#000000; stroke-width:8; stroke-opacity:1; fill:none\" points=\"\n",
       "  1457.81,297.355 1573.01,297.355 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 1592.21, 314.855)\" x=\"1592.21\" y=\"314.855\">y &lt; -0.3</text>\n",
       "</g>\n",
       "<circle clip-path=\"url(#clip0700)\" cx=\"1525.01\" cy=\"357.835\" r=\"21\" fill=\"#ff0000\" fill-rule=\"evenodd\" fill-opacity=\"1\" stroke=\"#000000\" stroke-opacity=\"1\" stroke-width=\"2.56\"/>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 1592.21, 375.335)\" x=\"1592.21\" y=\"375.335\">optimum</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip0700)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1260.12, 830.176)\" x=\"1260.12\" y=\"830.176\">somewhere here</text>\n",
       "</g>\n",
       "</svg>\n"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "contour( x,y,loss2d',\n",
    "         xlims = (1,5),\n",
    "         ylims = (-1,0),\n",
    "         legend = false,    \n",
    "         levels = 21,\n",
    "         title = \"Contour plot of loss function\",\n",
    "         xlabel = \"x\",\n",
    "         ylabel = \"y\",\n",
    "         annotation = (3.5,-0.7,text(\"somewhere here\",8)) )\n",
    "\n",
    "plot!([2.75,2.75],[-1,0.5],color=:red,linewidth=2,label=\"2.75 < x\",legend = true)\n",
    "plot!([1,5],[-0.3,-0.3],color=:black,linewidth=2,label=\"y < -0.3\")\n",
    "scatter!([2.75],[-0.75],color=:red,label=\"optimum\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The optimum is at (x,y) =      2.750    -0.750\n"
     ]
    }
   ],
   "source": [
    "lower = [2.75, -Inf]\n",
    "upper = [Inf, -0.3]\n",
    "\n",
    "Sol = optimize(fn2,lower,upper,[3.0,-0.5])\n",
    "printlnPs(\"The optimum is at (x,y) = \",Optim.minimizer(Sol))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Several Choice Variables:  Supplying the Gradient (extra)\n",
    "\n",
    "Supplying a function for calculating the derivatives improves speed and accuracy. See below for an example. (The `inplace=false` means that the function for the derivatives creates a new matrix at each call.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * Status: success\n",
      "\n",
      " * Candidate solution\n",
      "    Minimizer: [2.00e+00, -7.50e-01]\n",
      "    Minimum:   7.296963e-30\n",
      "\n",
      " * Found with\n",
      "    Algorithm:     L-BFGS\n",
      "    Initial Point: [1.00e+00, -5.00e-01]\n",
      "\n",
      " * Convergence measures\n",
      "    |x - x'|               = 9.34e-01 ≰ 0.0e+00\n",
      "    |x - x'|/|x'|          = 4.67e-01 ≰ 0.0e+00\n",
      "    |f(x) - f(x')|         = 8.75e-01 ≰ 0.0e+00\n",
      "    |f(x) - f(x')|/|f(x')| = 1.20e+29 ≰ 0.0e+00\n",
      "    |g(x)|                 = 2.13e-14 ≤ 1.0e-08\n",
      "\n",
      " * Work counters\n",
      "    Seconds run:   0  (vs limit Inf)\n",
      "    Iterations:    2\n",
      "    f(x) calls:    7\n",
      "    ∇f(x) calls:   7\n",
      "\n"
     ]
    }
   ],
   "source": [
    "function g2(x)                        #derivatives of fn2 wrt. x[1] and x[2]\n",
    "    G = [2*(x[1]-2), 2*4(4*x[2]+3)]   #creates a new vector: use inplace = false in optimize()\n",
    "    return G\n",
    "end\n",
    "\n",
    "Sol3 = optimize(fn2,g2,[1.0,-0.5],inplace=false)\n",
    "println(Sol3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "anaconda-cloud": {},
  "kernel_info": {
   "name": "julia-1.2"
  },
  "kernelspec": {
   "display_name": "Julia 1.4.0",
   "language": "julia",
   "name": "julia-1.4"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.4.0"
  },
  "nteract": {
   "version": "0.15.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
